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THE  VIEWS,  OPINIONS,  AND  FINDINGS  CONTAINED 
IN  THIS  REPORT  ARE  THOSE  OF  THE  AUTHORS  AND 
SHOULD  NOT  BE  CONSTRUED  AS  AN  OFFICIAL 
DEPARTMENT  OF  THE  AIR  FORCE  POSITION,  POLICY, 
OR  DECISION,  UNLESS  DESIGNATED  BY  OTHER 
OFFICIAL  DOCUMENTATION. 


PREFACE 


Today,  Program  Managers  (PM)  face  the  problem  of  acquiring 
more  complex  systems,  in  less  time,  without  sacrificing  system 
effectiveness,  readiness,  and  performance.  Groups  within  the  Air 
Force  have  recognized  the  need  to  provide  topical  guidance  to  PMs 
to  assist  them  in  meeting  this  responsibility.  Balancing 
Materiel  Readiness  Risks  and  Concurrency  in  Weapon  System  Acqui¬ 

sition:  A  Handbook  for  Program  Managers,  was  developed  to  offer 

advice  to  PMs  and  their  staffs  on  some  of  the  basic  pitfalls  to 
be  dealt  with  in  managing  materiel  readiness  in  the  presence  of 
concurrency. 

This  manual  addresses  the  problem  of  balancing  materiel 
readiness  risks  and  concurrency  by  considering  the  four  key 
issues  (or  problems)  the  PM  must  consider: 

•  the  difficulty  in  managing  for  materiel  readiness; 

•  the  implementation  of  concurrency  as  a  planning, 
management,  or  acquisition  strategy; 

•  the  integration  of  risk  analysis  in  program  planning 
and  management;  and 

•  the  development  of  strategies  for  pJanning,  managing, 
and  acquiring  a  system  in  light  of  these  factors. 

This  handbook  was  written  with  the  assumption  that  the 
reader  has  only  some  familiarity  with  weapon  system  acquisition 
program  management.  For  this  reason,  brief  overviews  of  program 
management,  scheduling  techniques,  and  risk  analysis  have  been 
provided.  However,  no  attempt  has  been  made  to  provide  a 
detailed  tutorial  on  these  subjects.  Where  appropriate,  more 


x 


detailed  guides  or  sources  of  information  on  these  subjects  are 
referenced  in  the  discussions.  In  addition,  listings  of  signifi¬ 
cant  information  sources  are  also  given. 

The  material  for  this  handbook  has  come  from  a  variety  of 
sources  which  fall  into  three  major  categories: 

•  DoD  guidebooks,  handbooks,  directives  and  instructions; 

•  studies  and  analyses  of  the  acquisition  process, 
program  management,  and  specific  weapon  system  acquisi¬ 
tions;  and 

•  interviews  with  various  program  office  and  contractor 
personnel,  as  well  as  others  with  program/acquisition 
experience. 

This  handbook  was  prepared  under  the  joint  sponsorship  of 
the  Air  Force  Business  Research  Management  Center  ( AFBRMC )  and 
the  Air  Force  Acquisition  Logistics  Center  ( AFALC/PTR ) ,  by 
Management  Consulting  &  Research,  Inc.  (MCR)  under  Contract 
Number  F33615-83-C-5050 .  It  could  not  have  been  written  without 
the  guidance  provided  by  AFBRfoC  and  AFALC/PTR.  MCR  also  received 
valuable  assistance  from  numerous  people  in  the  various  organiza¬ 
tions  contacted,  including  the  F-16  System  Program  Office  at 
Wright-Patterson  AFB ,  the  F-16  Program  Office  at  General 
Dynamics/Fort  Worth,  and  the  Ballistic  Missile  Office.  However, 
the  contents  and  ideas  in  this  handbook  are  not  intended  to 
reflect  Air  Force  policy  and  are  strictly  those  of  the  authors. 
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PART  I.  BACKGROUND  OF  THE  REQUIREMENT 


The  purpose  of  this  part  of  the  handbook  is  to  provide  the 
reader  with  an  understanding  of  why  this  handbook  was  developed, 
and  what  the  key  issues  are  to  which  the  handbook  responds.  This 
background  is  given  in  two  chapters .  Chapter  1  introduces  the 
handbook  and  provides  a  general  discussion  of  the  more  recent 
emphasis  on  system  or  materiel  readiness  as  an  acquisition 
priority  of  the  Program  Manager.  This  chapter  also  includes  a 

description  of  the  purpose,  scope,  and  organization  of  the  hand¬ 
book  . 

Chapter  2  addresses  the  major  kinds  of  issues  the  Program 
Manager  (PM)  must  address  in  dealing  with  the  problem  of  balanc¬ 
ing  materiel  readiness  risks  and  concurrency  in  a  program. 
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CHAPTER  1. 


INTRODUCTION 


•  Background/Purpose  of  the  Handbook 

•  Scope  of  the  Handbook 

•  Organization  of  the  Handbook 


1-i 


i 


■% 


CHAPTER  1 .  INTRODUCTION 


BACKGROUND/PURPOSE  OF  THE  HANDBOOK 


Today,  Program  Managers  (PM)  face  the  problem  of  acquiring 
more  complex  systems,  in  less  time,  without  sacrificing  system 
effectiveness,  readiness,  and  performance.  Balancing  Materiel 
Readiness  Risks  and  Concurrency  in  Weapon  System  Acquisition:  A 
Handbook  for  Program  Managers  results  from  a  need  to  provide  PMs 
with  topical  guidance  on  this  problem.  This  problem,  and  the 
need  to  address  it  effectively,  is  reflected  in  several  of  the 
initiatives  in  the  DoD  Acquisition  Improvement  Program.”^  These 
initiatives  are: 

o  Initiative  No.  9  -  System  Support  and  Readiness, 
establishing  readiness  objectives  for  each  weapon 
development  program  and  then  designing  in  reliability 
and  maintainability; 

•  Initiative  No.  11  -  Budgeting  for  Technological  Risk, 
evaluating,  quantifying  and  budgeting  for  technological 
risk; 

•  Initiative  No.  12  -  Test  Hardware  Funding,  requiring 
that  adequate  test  hardware  be  obtained  to  reduce 
overall  schedule  time  and  risks; 

•  Initiative  No.  16  -  Contractor  Incentives  for  Reliabi- 
1 ity  and  Support,  requiring  that  incentives  be  devel- 
oped  to  encourage  contractors  to  improve  reliability 
and  support ; 

•  Initiative  No.  21  —  Standardization  of  Operational  and 
Support  Systems,  requiring  that  development  and  use  of 
standard  operational  and  support  systems  achieve 
earlier  deployment  and  better  support  of  weapon  systems 
in  order  to  increase  force  readiness  and  support; 


— ^  Guidance  on  the  Acquisition  Improvement  Program  (AIP),  Deoutv 
Secretary  of  Defense  Memorandum,  8  June  1983. 
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Initiative  No.  30  -  Program  Manager  Control  Over  Loqis- 
t_lcs  and  Support  Funds,  requiring  that  anlj 

support  resources  be  shown  in  the  Service  POM  by  weapon 
system,  and  Program  Managers  be  given  more  control  of 
support  resources;  and 

•  Initiative  No.  31  -  Improved  Reliability  and  Support 
involving  improvement  of  reliability  and  support  for 
shortened  acquisition  cycle  programs. 

In  addition  to  the  Acquisition  Improvement  Program  (AIP), 
key  DoD  directives  emphasize  the  need  to  design  and  plan  for  sys¬ 
tem  readiness,  shortening  the  acquisition  cycle,  and  managing  the 
risks  in  acquiring  new  systems. Among  the  requirements 
included  in  these  directives  are  the  following: 

•  "Improved  readiness  and  sustainability  are  primary 
objectives  of  the  acquisition  process.  Resources  to 
achieve  readiness  will  receive  the  same  emphasis  as 
those  required  to  achieve  schedule  or  performance 
objectives.”  ( DoDD  5000.1) 

•  "These  resources  [to  achieve  readiness]  shall  include 
those  necessary  to  design  desirable  support  character¬ 
istics  intc  systems  and  equipment  as  well  as  those  to 
plan,  develop,  acquire,  and  evaluate  the  support." 

(DoDD  5000.39  ) 


Also,  as  directed  in  DoDD  5000.39  in  the  development  of 
acquisition  strategies,  programs  shall  emphasize: 


"  (1  ) 


i-cati°n  of  reliability  and  maintainability 
(R&M)  and  suppor tabi 1 ity  requirements. 


(2)  Evaluation  of  alternative  support  concepts  and  techni¬ 
ques  to  minimize  cost  and  support  risks. 

(3)  Test  articles  to  support  R&M  and  ILS  development,  test, 
and  evaluation  ( DT&E ) . 


y  DoD  Directive  5000.1,  Major  System  Acquisitions.  Under 

Secretary  of  Defense  (Research  and  Engineering),  29  March  1982. 

DoD  Directive  5000.39,  Acquisition  and  Management  of  Integrated 

k-?9istlc  Support _ for  Systems  and  Equipment,  Assistant  Secretary 

of  Defense  (Manpower,  Reserve  Affairs  and  Logistics),  17  November  1983. 
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(4)  The  early  establishment  of  system  readiness  and  sup- 
portability  thresholds  for  verification  or  assessment 
during  test  and  evaluation  (T&E)  before  decision  mile¬ 
stones  .  " 

DoDD  5000.39  also  states  that  early  ILS  planning  shall  be 
based  on  "...documented  logistic  support  analyses  (LSA)  to  link 
design  and  ILS  requirements  to  system  readiness  thresholds  and  to 

f 

define  detailed  support  element  requirements L " 

At  the  same  time  there  has  been  a ■ significant  increase  in 

emphasis  on  designing  and  acquiring  systems  that  can  be  "ready 

when  needed."  There  has  also  been  an  increase  in  concern  with 

the  significant  increases  in  the  time  it  takes  to  acquire  major 
3/ 

weaPon  systems.  In  1977 ,  the  Defense  Science  Board  examined 

the  growth  in  the  duration  of  the  time  required  to  acquire  new 

weapon  systems  and  found  that: 

"...the  front  end  period  from  initial  program  con¬ 
ception  to  DSARC  II  has  increased  substantially  - 
from  less  than  two  years  in  the  1950's  to  an 
average  of  nearly  five  years  at  present....  [The 
Task  Force]  also  concluded  that  the  production  and 
deployment  period  has  increased  considerably  in 
recent  years  as  a  result  of  such  pressures  as 
operational  test  and  evaluation,  reduced  concur¬ 
rency  and  production  stretchouts  .... "£/ 

Since  that  report  was  written,  the  use  of  concurrency  as  an 

acquisition  strategy  has  once  more  become  acceptable  as  a  means  of 


3  /  • 

— '  While  the  emphasis  in  this  handbook  is  on  major  weapon  systems, 
(i.e.,  those  requiring  DSARC  approval),  many  of  these  concerns 
also  face  and  are  equally  applicable  to  less  than  major  systems 
and  programs • 

—!  Report  of  the  Acquisition  Cycle  Task  Force  -  1977  Summer 

£tudy,  Defense  Science  Board,  Office  of  the  Under  Secretary 

of  Defense  (Research  and  Engineering),  15  March  1978. 
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reducing  acquisition  time.-/  However,  as  always,  applying 
concurrency  to  a  program  creates  problems  in  its  own  right,  over 
and  above  those  already  facing  the  Program  Manager.  For  example, 
there  are  risks  associated  with  meeting  readiness  requirements 
goals  in  acquisitions  where  concurrency  is  being  applied  to 
reduce  the  time  to  field  the  system. 

This  handbook  addresses  the  concerns  facing  Program  Managers 
in  balancing  the  weapon  system's  readiness  requirements,  and  the 
risks  associated  with  meeting  these  requirements  goals,  in  acqui¬ 
sitions  with  concurrency.  Its  purpose  is  to  raise  questions  and 
identify  concerns  of  which  the  Program  Manager  should  be  aware 
and  for  which  he  should  plan.  Guidance  is  provided,  in  areas 
where  no  guidance  previously  existed,  to  assist  the  Program 
Manager  in  program  planning. 

*  SCOPE  OF  THE  HANDBOOK 

This  handbook  is  designed  to  be  used  by  Program  Managers  and 
their  staffs  in  formulating  decisions  concerning  the  basic  pro¬ 
gram  acquisition  strategy  and  management  priorities.  As  such,  it 
was  developed  with  the  fundamental  assumption  that  the  reader  is 
familiar  with  the  weapon  system  acquisition  process  and  the  basic 


For  a  brief  history  of  the  ups  and  downs  in  the  history  of  con¬ 
currency  m  various  system  acquisitions  see:  Gibson,  Robert  G., 
Concurrency, "  Defense  Systems  Management  Review  -  Defense 

Acquisition: - The_  Process  and  the  Problems.  Defend 

Management  College,  Autumn  1979;  and  Harvey,  Thomas  E.,  "Concur¬ 
rency  Today  in  Acquisition  Management,"  Defense  Systems  Manaqe- 
lEg.rct  Review  Defense  Acquisition:  Issues  and  Answers.  Numh^r  1, 
Volume  1,  Defense  Systems  Management  College,  Winter  1980. 
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responsibilities  of  the  various  organizations  involved  in  design¬ 
ing  and  acquiring  Air  Force  systems  (i.e.,  Air  Force  Systems 
Command,  Air  Force  Logistics  Command,  Air  Force  Acquisition 
Logistics  Center,  etc.) 

The  reasons  behind  this  assumption  are  threefold: 

•  It  is  not  the  intention  of  this  handbook  to  be  redun¬ 
dant  and  restate  information  already  available  in 
other  standard  sources,  except  in  the  context  of  sup¬ 
porting  specific  points  of  discussion. 

•  There  are  numerous  sources  already  available  which  are 
designed  to  provide  detailed  discussions  as  well  as 
t'ssic  understanding  of  specific  elements  of  program 
management,  system  engineering,  logistic  support  anal¬ 
ysis,  etc.  These  sources  clearly  state  organizational 
and  institutional  positions  with  respect  to  the  topics 
they  address.  They  have  been  designed  to  relate  fac¬ 
tual  information  concerning  roles,  responsibilities, 
organizational  relationships  and  procedures,  and  are 
essentially  tutorial  in  nature.  Lists  of  major  sources 
are  given  by  topic  in  Appendix  A. 

•  This  handbook  is  designed  to  raise  issues  and  to  be 
essentially  cautionary  in  nature.  It  is  not  intended 
to  provide  a  comprehensive  discussion  of  the  various 
topics  addressed,  since  to  attempt  to  do  so  would 
almost  assuredly  mean  the  inadvertant  exclusion  of  a 
relevant  example  or  approach.  Rather,  to  the  degree 
possible,  it  is  a  compendium  of  "things  the  Program 
Manager  should  think  about." 

Given  these  constraints,  this  handbook  has  been  designed  to 
focus  on  the  four  key  issues  (or  problems)  the  Program  Manager 
must  consider  in  balancing  materiel  readiness'  risks  and  concur¬ 
rency.  These  are: 

•  the  definition  of  materiel  readiness  risks,  the  diffi¬ 
culty  in  managing  materiel  readiness,  and  the  factors 
(i.e.,  activities,  organizations  and  events)  in  the 
program  which  most  directly  relate  to  materiel  readi¬ 
ness; 

•  the  definition  of  concurrency,  and  the  techniques  for 
scheduling  program  activities,  managing  the  imposition 
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of  concurrency  on  a  schedule,  and  analyzing 
associated  with  applying  concurrency; 


the  risks 


the  integration  of  risk  analysis  in  management, 
conduct  or  risk  analyses,  and  the  various  tools 
can  use  to  perform  risk  analysis;  and 


the 

the  PM 


•  the  development  of  strategies  for  planning,  managing 
and  acquiring  a  system  in  light  of  these  factors. 

These  topics  are  first  addressed  in  terms  of  how  they  are  prob¬ 
lems  the  PM  must  manage.  They  are  then  each  discussed  in 
individual  sections  of  the  handbook.  These  sections  contain 
brief  discussions  of  basic  information  the  PM  and  his  staff 
should  be  aware  of  in  considering  these  issues,  and  suggested 
techniques  for  addressing  them. 


ORGANIZATION  OF  THE  HANDBOOK 


This  handbook  is  divided  into  five  major  parts,  each  of 
which  is  divided  into  chapters  addressing  specifically-related 
topics.  A  set  of  appendices  follows  the  text.  The  appendices 
contain  more  detailed  reference  information  supporting  or  ela- 
borating  on  the  chapter  information. 

The  five  parts  of  the  handbook  are  designed  to  be  self-con¬ 
tained  discussions  of  different  aspects  of  the  Program  Manager's 
problem.  The  reader  can  read  all  or  only  relevant  parts  of  the 
handbook,  depending  on  his  or  her  interest,  experience,  and 
specific  program  responsibilities. 

Tne  five  parts  of  the  handbook  are  designed  to  address  the 
major  aspects  of  concern  to  the  PM  in  managing  the  problem  of 
balancing  materiel  readiness  risks  and  concurrency  in  a  system 
acquisition.  These  aspects  can  be  thought  of  as  the  following: 
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Defining  the  problem  and 
which  the  PM  needs  to  be 
examines  the  background 
the  emphasis  on  materiel 
PM  must  contend  with  in 


determining  the  issues 
aware.  Part  I  of  the 
of  the  requirement  to  i 
readiness  and  major  is 
managing  for  readiness. 


handbook 
ncrease 
sues  the 


Determining  the  nature  of  materiel  readiness,  in  terms 
of  a  capability  resulting  from  specific  activities. 
Part  II  focuses  on  the  structure  of  the  acquisition 
process,  discussing  the  activities  and  functions  in  a 
system  acquisition  with  the  greatest  bearing  on 
materiel  readiness. 


Considering  concurrency  as  a  scheduling  and  management 
problem.  Part  III  examines  basic  scheduling  techniques 
the  PM  uses,  the  analysis  of  the  impact  of  concurrency 
on  the  schedule  and  how  the  schedule  risks  associated 
with  concurrency  can  be  evaluated. 

Integrating  risk  analysis,  assessment  and  management  to 
assist  in  balancing  materiel  readiness  and  concurrency. 
Part  IV  examines  the  basic  elements  of  conducting  a 
risk  analysis  and  how  such  analysis  can  be  integrated 
to  manage  risks.  Part  IV  also  examines  basic  risk 
analysis  tools  and  techniques. 


Developing  an  acquisition  strategy  to  balance  these 
elements  in  the  context  of  the  program1 s  goals  and  con¬ 
straints.  Part  V  summarizes  the  problems  and  concepts 
discussed  in  the  preceeding  parts  of  the  handbook  and 
suggests  strategies  for  addressing  such  problems. 


Exhibit  1-1  shows  the  relationships  among  the  major  areas  of 
concern  and  the  remaining  contents  of  the  handbook.  Part  I, 


BACKGROUND  OF  THE  REQUIREMENT,  contains  two  chapters.  Following 
this  introductory  chapter,  Chapter  2  discusses  the  factors  the 
Program  Manager  must  consider  in  managing  materiel  readiness  and 
its  related  risks,  and  concurrency. 


Part  II,  PROGRAM  FACTORS  AND  RELATIONSHIPS,  addresses  those 
elements  in  the  program  that  relate  to  materiel  readiness,  empha¬ 
sizing  potentially  concurrent  relationships  among  those  activi¬ 


ties  as  well  as  with  other  program  activities,  functions  and 
areas.  It  is  composed  of  three  chapters: 
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Part  I.  Background  of  the  Requirement 
Chapter  1.  Introduction 
Chapter  2.  Factors  to  Consider 


©  Definition  of  Terms 

-  Materiel  Readiness 

-  Concurrency 


Part  II. 


< 


•  Managing  for  Readiness 


•  Concurrency  as  the 
Program  Manager's 
Dilemma 


r 

Part  III. 
h! 


•  Integration  of  Risk 

Analysis  and  Management 


{ 


Part  IV. 


Program  Factors  and  Relationships 
Chapter  3.  Overview  of  the  Systems  Acquisition 
Process 

Chapter  4.  Factors  Influencing  Materiel 
Readiness 

Chapter  5.  Concurrency-Related  Factors 
Appendix  C.  Summary  of  Major  Acquisition 
Activities 

Appendix  D.  Additional  Logistic  Support  Analysis 
Information 

Scheduling  Techniques  and  Concurrency  Analysis 
Chapter  6.  Summary  of  Scheduling  Techniques 
Chapter  7.  Structure  of  Concurrency  Scheduling 
Analysis 

Chapter  8.  Analysis  of  Concurrency  Schedule  Risk 
Appendix  E.  Guidance  on  Developing  a  Schedule 
Network 

Overview  of  Risk  Analysis 

Chapter  9.  Elements  of  Risk  Analysis 

Chapter  10.  Risk  Analysis  Alternatives 

-> - - - - - / 


Part  V. 


Planning,  Management,  and  Acquisition  Strategies 
Chapter  11.  Suggestions  for  Strategies 
Appendix  A.  References 

Appendix  B.  Glossary  of  Terms  and  Abbreviations 


Exhibit  1-1.  RELATIONSHIPS  AMONG  HANDBOOK  CONTENTS 


•  Chapter  3  is  a  brief  overview  of  the  Air  Force  weapon 
system  acquisition  process,  with  a  discussion  of  the 
relevant  OSD  and  Air  Force  regulations,  key  organiza¬ 
tions,  and  major  program  activities; 

•  Chapter  4  is  a  more  detailed  discussion  of  those 
elements  in  a  weapon  system  acquisition  program  that 
directly  relate  to  materiel  readiness;  and 

•  Chapter  5  examines  the  acquisition  activities  in  light 
of  potential  concurrency  relationships  and  discusses 

the  activities  particularly  related  to  materiel  readiness. 

Part  III,  SCHEDULING  TECHNIQUES  AND  CONCURRENCY  ANALYSIS, 
contains  three  chapters; 

•  Chapter  6  provides  a  brief  overview  of  major  schedule 
analysis  techniques  generally  used  in  program  schedule 
planning  and  analysis; 

•  Chapter  7  describes  a  basic  analytical  approach  for 
analyzing  schedules  in  order  to  apply  concurrency;  and 

•  Chapter  8  describes  the  conduct  and  application  of  a 
concurrency  schedule  risk  analysis. 

Part  IV,  OVERVIEW  OF  RISK  ANALYSIS,  provides  a  general 
discussion  of  risk  analysis.  It  has  two  chapters: 

•  Chapter  9  in  which  the  elements  of  risk  analysis  are 
described;  and 

•  Chapter  10  in  which  the  various  alternatives  for  per¬ 
forming  risk  analyses  are  discussed. 

Part  V,  MANAGEMENT  AND  ACQUISITION  STRATEGIES,  is  the  final 
part  of  the  handbook.  In  the  final  chapter  (Chapter  11),  various 
planning,  management,  and  acquisition  strategies  are  described. 

In  addition  to  these  major  discussions,  there  are  five 
appendices.  Appendix  A  contains  a  listing  by  topic  of  pertinent 
references  of  potential  interest  to  Program  Managers  and  their 
staffs.  Appendix  B  is  a  Glossary  of  Terms  and  Abbreviations  used 
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in  this  handbook.  Appendix  C  includes  a  summary  of  major  Air 
Force  system  acquisition  activities.  Appendix  D  has  additional 
information  on  Logistic  Support  Analysis.  Appendix  E  provides 
basic  guidance  on  developing  a  schedule  network. 
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CHAPTER  2.  FACTORS  TO  CONSIDER 


Definition  of  Terms 
Managing  for  Readiness 

Concurrency  as  the  Program  Manager's  Dilemma 
Integration  of  Risk  Analysis  and  Management 


CHAPTER  2.  FACTORS  TO  CONSIDER 


The  Progam  Manager  who  must  balance  materiel  readiness  risks 
in  system  acquisitions  with  concurrency  faces  a  variety  of  diffi¬ 
culties,  not  the  least  of  which  is  confusion  over  what  is  meant 
by  "materiel  readiness"  and  "concurrency".  In  addition  to  these 
semantic  difficulties,  there  are  built-in  difficulties  associated 
with  managing  for  a  particular  system  capability,  such  as  readi¬ 
ness.  This  is  particularly  true  in  an  environment  in  which  con¬ 
currency  is  being  used  as  part  of  the  acquisition  strategy. 
Related  issues,  such  as  the  priority  of  system  support  in  the 
program,  the  use  of  planning  and  scheduling  techniques,  the  inte¬ 
gration  of  risk  analysis  and  management  and  the  management  phil¬ 
osophy  or  strategy,  all  seriously  influence  the  difficulty  the 
Program  Manager  has  in  effectively  balancing  program  concerns. 
This  chapter  is  intended  to  briefly  explore  some  of  the  major 
considerations  the  PM  must  face  in  balancing  materiel  readiness 
risks  and  concurrency.  While  these  are  not  all  of  the  issues  the 
PM  must  contend  with  in  managing  for  materiel  readiness,  they 
should  be  viewed  as  the  starting  point  from  which  one  should 
explore  a  program  to  consider  other  impacts. 

In  addition  to  the  internal  constraints,  there  is  the  out¬ 
side  world,  as  illustrated  in  Exhibit  2-1.  All  programs  are 
vulnerable,  to  varying  degrees,  to  impacts  from  political,  social 
and  economic  conditions  over  which  the  Program  Manager  has  no 
control.  While  these  are  real  probiems,  techniques  and  sugges¬ 
tions  for  dealing  with  these  areas  are  outside  the  scope  of  this 
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Source : 


Adapted  from  the  Navy  Program  Manager's  Guide. 
Headquarters,  Naval  Material  Command,  July  1983 
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Exhibit  2-1.  EXTERNAL  INFLUENCES  ON  PROGRAM 
MANAGEMENT  DECISIONS 
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handbook.  The  internal  concerns,  however,  are  of  interest.  The 
major  issues  concerning  the  Program  Manager's  ability  to  balance 
materiel  readiness  risks  and  the  requirements  of  concurrency  are 
briefly  discussed  below.  These  issues  are: 

•  the  definition  of  the  terms  "materiel  readiness"  and 
"concurrency", 

•  the  problems  associated  with  managing  materiel  readi¬ 
ness, 

•  the  problems  of  managing  concurrency,  and 

•  the  integration  of  risk  analysis  and  program  manage¬ 
ment  . 

DEFINITION  OF  TERMS 

In  examining  the  literature,  policy  directives  and  regula¬ 
tions,  and  in  interviews  with  OSD,  Air  Force  and  contractor 
personnel,  a  basic  source  of  confusion  is  revealed  concerning 
the  meaning  of  certain  terms.  Depending  on  the  perspective, 
"materiel  readiness"  and  "concurrency"  may  mean  substantially 
different  things  to  different  people. 

1 .  Materiel  Readiness 

While  the  term  "readiness"  is  frequently  used,  it  is 

rarely  defined.  Direction  on  Logistic  Support  Analysis  (LSA) 

1  / 

refers  to  readiness  drivers.- 1 '  DoDD  5000.1  states  that  "system 
readiness  is  a  primary  objective  of  the  acquisition  process,"  but 
does  not  define  what  is  meant.  DoDD  5000.39  gives  the  following 
definition  for  System  Readiness  Objective: 

A/  Logistic  Support  Analysis,  MIL-STD-1388-1A,  11  April  1983. 
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.  .  A  criterion  for  assessing  the  ability  of  a  system  to  under- 
t  e  and  sustain  a  specified  set  of  missions  at  planned  peacetime 
an  war  ime  utilization  rates.  System  readiness  measures  take 
explicit  account  of  the  effects  of  system  design  R&M ,  the  charac- 
teristics  and  performance  of  the  support  system,  and  the  quantity 
and  location  of  support  resources.  Examples  of  system  readiness 

trS  combat  sortie  rate  over  time,  peacetime  mission 
p  ble  rate,  operational  availability,  and  asset  ready  rate.” 


The  DoD  Dictionary  of  Military  and  Associated  Terms  gives  the 
following  definition  of  Operational  Readiness:-^ 


"The 
equipment 
organized 
express  a 


capability  of  a  unit/formation,  ship,  weapon  system  or 
to  perform  the  missions  or  functions  for  which  it  is 
or  designed.  May  be  used  in  a  general  sense  or  to 
level  or  degree  of  readiness." 


The  problem  of  defining  readiness  has  become  even  more  sig¬ 
nificant  since  the  recent  policy  emphasizing  system  readiness  as 
being  of  equal  importance  as  cost,  schedule  and  performance  (DoDD 
5000.1  and  5000.39).  Interpretation  of  the  term  tends  to  focus 
on  either  of  two  directions:  a  very  specific  measurement  of 
capability  which  can  be  quantified,  or  a  more  general  area  cover¬ 
ing  a  group  of  capabilities  or  elements.  The  first  interpreta¬ 
tion  tends  to  focus  on  a  single  parameter  or  set  of  parameters 
which  best  typifies  the  readiness  of  a  system.  As  noted  in  a 
discussion  of  this  issue,  this  has  tended  to  center  on  the  calcu- 
lation  of  Operational  Avai labi 1 ity 


Department  of  Defense  Dictionary  of  Military  and  Associated 
Terms,  JCS  Pub.  1,  The  Joint  Chiefs  of  Staff,  1  June  1979. 


1/ 


Brabson,  Col.  G.  Dana,  and  Solomond,  Dr. 
Coequal",  Concepts:  Special  Issue  -  The 
ment  Program,  Volume  5.  Mumhpr  3r 
College,  Summer  1982. 


John  P.,  "Readiness  - 
DoD  Acquisition  Improve- 

Systems  Management 
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"There  is  a  desire  to  focus  on  a  single  parameter  if  possi¬ 
ble,  because  this  would  facilitate  intercomparisons  and  simplify 
the  overall  process.  As  of  this  writing,  there  is  a  trend  toward 
the  use  of  operational  availability  (A0)  as  that  single  parame¬ 
ter.  In  as  much  as  operational  availability  is  expressed  as  the 
ratio  of  uptime  divided  by  total  time,  it  is  a  measure  both  of 
the  field  reliability  and  the  supportabil  ity  of.  the  hardware." 

The  interpretation  of  operational  availability,  as  repre¬ 
sentative  of  a  system's  readiness,  is  desirable  for  certain  types 

f  )  • 

of  analysis  and  planning,  in  that  A0  does  represent  the  collec¬ 
tive  productivity  of  the  support  system.  However,  it  has  its  own 
associated  difficulty  due  to  differences  in  how  availability  is 
interpreted.  Using  this  measure  also  tends  to  be  too  restric¬ 
tive,  for  the  purposes  of  achieving  the  goals  of  this  handbook. 
Finally,  it  is  not  practical  to  try  to  manage  a  program  to  a 
single  target  quantity  such  as  operational  availability. 

The  second  interpretation  of  system  readiness  is  oriented 
toward  a  more  general  consideration  of  the  system  support  struc¬ 
ture,  as  indicated  by  the  DoDD  5000..39  definition  of  System 
Readiness  Objective.  Readiness  is  viewed  as  the  capability  to 
not  only  operate  the  primary  system,  but  also  to  have  in  place  a 
full  capability  to  maintain  the  system.  This  includes  having  the 
necessary  support  equipment,  diagnostic  equipment,  adequately 
trained  personnel,  complete  technical  documentation,  etc.  Most 
of  these  capabilities  are  captured  under  the  heading  of  logistic 
support.  (See  Appendix  D  for  definitions  of  Integrated  Logistics 
Support  elements. ) 

In  this  handbook,  the  more  general  interpretation  of  materiel 
readiness,  as  represented  by  the  logistic  support  system,  has  been 
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used.  The  major  reason  for  this  is  that  it  allows  for  discus¬ 
sions  of  specific  program  activities  and  events  which  must  be 
planned  and  monitored.  Major  areas  of  analysis  such  as  manpower 
and  personnel,  training  and  training  devices,  and  support  equip¬ 
ment  can  be  considered  in  terms  of  specific  activities  within  the 
area,  and  how  activities  relate  among  different  areas. 

2 .  Concurrency 

The  term  "concurrency"  also  presents  h  source  of  con¬ 
fusion  for  similar  reasons.  In  the  context  of  the  weapon  system 
acquisition  process,  concurrency  has  often  been  defined  in  a  very 
restrictive  sense  as:  "The  conduct  of  the  steps  leading  to  pro¬ 
duction  for  inventory  before  the  end  of  the  full-scale  develop¬ 
ment  time  span."—,/  In  examining  the  literature  and  in  interviews 
with  SPO  and  contractor  personnel,  several  other  interpretations 
come  to  light,  including: 

•  parallel  (back-up)  technological  development; 

•  simultaneous,  but  independent,  subsystem  development 

and  testing ; 

•  co-production;  and 

•  overlap  of  dependent,  normally  sequential  activities. 

Each  of  these  interpretations  is  acceptable;  however,  not 

all  are  appropriate  for  consideration  in  this  handbook.  As  in 
the  case  of  the  definition  of  materiel  readiness,  the  least 
restrictive  definition  has  been  used  in ' d iscussing  concurrency: 
the  overlapping  of  dependent,  normally  sequential  activities. 

g.ePort  of  the  Acquisition  Cycle  Task  Force:  1977  Summer  Study, 

Defense  Science  Board,  15  March  1978.  - - - — 
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Using  this  definition  allows  for  the  consideration  of  a  greater 
variety  of  the  problems  facing  Program  Managers  in  managing  con¬ 
currency.  Activities  in  each  of  the  Design  and  Development  Phases 
can  be  considered,  rather  than  only  those  in  the  Full  Scale 
Development  and  Production  Phases. 

Another  aspect  in  defining  concurrency  involves  the  recogni¬ 
tion  that  there  are  different  types  of  concurrency.  A  single 
program  may  use  the  concept  of  concurrently  scheduling  activities 
for  a  number  of  different  reasons,  in  response  to  different 
requirements.  One  way  of  categorizing  the  different  types  of 
concurrency  is: 

•  normal  concurrency,  used  as  a  scheduling  strategy  for 
balancing  workload  and  personnel  assignments; 

•  planned  concurrency,  used  as  an  acquisition  strategy 
for  allowing  an  early  IOC  or  for  reducing  risk;  and 

•  exceptional  concurrency,  used  as  a  management  strategy 
to  compensate  for  unforeseen  problems  or  to  respond  to 
a  crisis. 

Normal  concurrency  refers  to  the  strategy  of  scheduling 
activities  to  maximize  smooth  workload  distribution  and  minimize 
personnel  assignment  turbulence  and  peaks  and  valleys  in  accom¬ 
plishing  tasks.  Most  programs  are  planned  to  achieve  smooth, 
continuous  progress  by  a  planned  set  of  tasks  designed  to  support 
specific  progress  milestones.  These  milestones  frequently 
involve  status  reviews  and  go/no  go  decisions. 

In  a  completely  sequential  schedule,  involving  no  concur¬ 
rency,  activities  in  a  phase  initiated  by  these  decision  mile¬ 
stones  should  not  be  started  until  after  the  decision  has  been 
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finalized. 


In  reality,  such  an  approach  would  mean  the  cessation 
of  work  while  a  decision  is  beihg  made  and  the  reassignment  of 
personnel  to  other  productive  tasks,  if  possible.  This  is  a 
costly  requirement  which  introduces  unnecessary  and  undesirable, 
confusion  as  well  as  resulting  in  inefficient  use  of  resources 
(personnel  and  money).  What  is  usually  done  on  programs  (as  well 
as  smaller  projects)  is  to  schedule  continuous  activities  in  the 
areas  affected  by  these  decisions.  These  tasks  may  be  catch-up/ 
clean-up  tasks,  such  as  documentation;  or  they  may  involve 
preliminary  planning  and  research  for  the  following  phase.  In 
either  case,  concurrency  is  used  as  a  schedule  management  technique 
Pj_anned  concurrency  refers  to  the  use  of  concurrency  as  a 
technique  for  reducing  risk  or  achieving  an  earlier  than  normal 
initial  operating  capability  (IOC).  Concurrency  has  been  used  in 
the  F-16  program,  for  example,  as  a  risk  management  technique. 
Initial  delivery  dates  for  the  F-16  C/D  configurations  have  been 
set  in  order  to  allow  for  a  longer  test  and  evaluation  period; 
and  to  allow  for  some  latitude  in  schedule  should  problems  arise. 
This  has  meant  concurrently  scheduling  some  of  the  system's 
integration  tasks,  but  with  the  intention  of  protecting  the  ulti¬ 
mate  Air  Force  delivery  date.  This  type  of  concurrency  is  used 
in  many  different  ways  throughout  programs  to  reduce  risks  of  not 
meeting  a  key  schedule  milestone* 

The  other  major  application  of  planned  concurrency  is  as  a 
means  of  achieving  an  earlier  IOC  than  would  usually  be  achieva- 
ble ,  using  conventional  program  planning.  Among  large  programs 
which  have  used  this  technique  are  the  Air  Force's  F-16  and 
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Peacekeeper  (MX)  programs,  and  the  Army's  Multiple  Launch  Rocket 
System  (MLRS).  In  these  programs,  the  acquisition  strategy  has 
been  designed  to  have  overlapping  activities  or,  in  the  case  of 
the  MLRS,  the  elimination  of  a  whole  design  phase  (the  Demonstra¬ 
tion  and  Validation  Phase).  Management  safeguards  have  been 
built  into  the  program  planning  in  the  form  of  more  intensive 
status  monitoring  techniques,  reporting  systems  and  corrective 
action  systems.  While  this  concurrency  is  planned,  it  is  not 
"normal"  since  it  requires  the  use  of  additional  planning  and 
management  strategies  in  order  to  not  incur  unacceptable  risks  to 
the  programs. 

Exceptional  concurrency  refers  to  circumstances  under  which 
concurrency  is  used  as  a  crisis  response.  Normal  and  planned 
concurrency  can  be  considered  as  crisis  avo- dance  techniques. 
Exceptional  concurrency  is  applied  in  situations  where,  for  one 
reason  or  another,  the  original  program  plans  must  be  changed 
part  way  through  the  program  development.  Changes  in  IOC,  fund¬ 
ing,  system  performance  requirements  or  schedule  slippage  may 
force  a  Program  Manager  into  concurrently  scheduling,  or  elimin¬ 
ating,  tasks  which  would  normally  not  be  scheduled  or  planned  in 
such  a  way.  Concurrency  applied  after  the  fact  (i.e.,  after  the 
initial  program  plans  and  schedules  have  been  developed)  can 
present  unavoidable  management  problems.  (These  will  be  con¬ 
sidered  further  later  in  this  chapter. )  It  is  important  to 
recognize  these  differences  in  the  types  and  reasons  for  applying 
concurrency,  since: 
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•  It  is  easy  to  not  recognize  when  one  has  created  a 
concurrent  situation,  with  all  of  the  potential  advan- 
tages  and  disadvantages* 

•  ^  i^Por tant  to  recognize  that  not  all  concurrency 

i s  ba  d" . 

Finally,  concurrency  must  be  considered  in  terms  of  its 
magnitude.  This  refers  to  the  quantity,  quality  or  site  of  the 
concurrency.  Concurrency  can  be  applied  in  different  amounts, 
i • e ♦ ,  amount  of  time  the  activities  overlap.  Concurrency  can 
also  be  applied  at  different  levels  (or  sites)  within  the 
program,  both  within  a  given  function  or  across  functions. 

The  Program  Manager  should  recognize,  then,  that  in  con¬ 
sidering  concurrency  and  its  application  in  his  or  her  program, 
it  should  be  considerd  in  terms  of: 

•  definition, 

•  type,  and 

•  magnitude. 

In  addition  to  the  problem  of  defining  materiel  readiness 
and  concurrency,  the  PM  must  also  develop  a  strategy  for  trying 

to  optimize  the  system's  ultimate  readiness  in  light  of  other 
program  goals. 

MANAGING  FOR  READINESS 


The  system  acquisition  process  has  evolved  over  time,  with 
priorities  shifting  from  emphasizing  system  performance,  to 
reducing  acquisition  costs  (sometimes  at  the  expense  of  total 
life  cycle  costs),  to  reducing  acquisition  time,  to  designing  to 

-  f  S ..  * 
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increased  system,  or  materiel,  readiness.  In  effect,  what  has 
happened  has  been  that  with  each  new  priority,  an  additional 
requirement  has  been  levied  on  the  Program  Manager  to  monitor 
and  plan  for  another  priority  -  somewhat  similar  to  having  to 
juggle  four  balls  instead  of  the  previous  three. 

The  Program  Manager  is  responsible  for  managing  for  system 
readiness,  developing  readiness  objectives,  controlling  logistics 
and  support  resources,  analyzing  readiness  drivers,  monitoring 
the  status  of  the  design  development  decisions  to  assure  that 
readiness  is  considered,  and  reporting  on  the  supportability 
characteristics  and  requirements  of  the  design.^ 

The  activities  that  influence  a  system's  readiness  are  dis¬ 
tributed  among  several  program  areas.  In  addition  to  Integrated 
Logistic  Support/Logistic  Support  Analysis,  activities  that  fall 
under  the  Systems  Engineering  area,  particularly  Specialty 
Engineering,  also  significantly  influence  readiness.  Specialty 
Engineering  collectively  covers  "those  disciplines  which  support 

the  design  process  by  applying  knowledge  from  a  specific  area  to 

6  / 

ensure  system  operability  in  its  operational  environment."-' 
Specialized  engineering  disciplines,  such  as  reliability,  main¬ 
tainability,  human  and  parts  engineering,  transportability, 
safety,  and  electromagnetic  compatibility,  may  start  out  in  the 


V  See  DoDD  5000.1,  DoDD  5000.39  and  MIL-STD-1388-1A  for  the  policy 
relating  to  these  responsibilities,  and  "Readiness  -  Coequal"  by 
Brabson  and  Solomond  (Concepts,  Vol .  5,  No.  3,  Summer  1982) 
for  a  discussion  and  analysis  of  these  responsibilities. 

— /  System  Engineering  Management  Guide,  Defense  Systems  Management 
College,  3  October  1983. 
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early  stages  of  the  program  being  collectively  directed  under  the 
single  function,  or  program  area,  of  Systems  Engineering.  As  the 
program  matures,  many  of  these  activities  will  be  separated  into 
individual  program  areas,  such  as  ILS,  while  others  will  stay  in 
the  System  Engineering  program  area. 

Other  program  areas,  such  as  Program  Control  and  Business 
Management,  play  key  roles  in  supporting  the  readiness  management 
responsibilities  of  the  Program  Manager.  These  areas  are  respon¬ 
sible  for  monitoring  and  reporting  on  the  status  of  the  master 
schedule  and  the  subschedules. 

This  distribution  of  responsibilities  produces  numerous 
management  demands  in  order  to  give  system  readiness  adequate 
exposure.  Horror  stories  abound  of  decisions  made  during  the 
acquisition  process  concerning  the  cancellation  of  apparently  low 
priority  items  which  were  ultimately  recognized  as  long  leadtime, 
critical  support  equipment.  Decisions  were  made  because: 

•  the  logistics  impacts  were  not  considered,  or  recoa- 

nized;  and  y 

•  logistics  support  has  historically  not  been  a  high- 
priority  item. 

This  latter  point  is  evidenced  by  the  historically  frequent 
deferral  of  ILS/LSA  requirements.  Program  office  personnel 
involved  in  logistics  have  repeatedly  expressed  concern  that 
because  much  of  the  substantive  logistics  analyses  and  decisions 
have  been  made  later  in  the  program,  logistics-related  funds  have 
tended  to  be  vulnerable  to  being  used  for  other  purposes.  This 
results  in  development  of  elements  of  log istics ' support ,  such  as 
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technical  documentation  and  support  equipment,  being  postponed 
until  very  late  in  the  development  process  or  possibly  until 
after  fielding  of  the  system. 

While  increased  emphasis  on  ILS  should  reduce  some  of  the 
vulnerability  of  acquisition  logistics  resources,  it  will  still 
be  difficult  to  focus  concentration  on  the  diversified  activities 
which  relate  to  readiness.  The  ability  to  do  this  occurs  on  a 
program-by-program  basis  since  efforts  are  still  underway  to 
develop  institutional  processes  such  those  discussed  in 
MIL-STD-1388-1A. 

In  addition  to  the  problem  of  distributed  activities  relat¬ 
ing  to  materiel  readiness,  there  is  another  fundamental  problem 
in  managing  materiel  readiness  risks,  and  that  is  relating 
materiel  readiness  to  risk  analysis.  As  discussed  above,  activi¬ 
ties  impacting  the  ability  to  produce  a  materielly  "ready"  system 
are  distributed  in  many  areas  within  the  SPO  and  contractor. 

This  in  itself  makes  it  difficult  to  coordinate  these  activities 
in  a  coherent  plan  for  focusing  on  system  readiness.  Relating 
the  status  of  these  activities  in  a  way  that  facilitates  risk 
assessment  becomes  a  real  problem  for  the  Program  Manager. 

Generally  speaking,  there  are  a  number  of  different  types  of 
risk  that  may  be  considered  in  program  management.  The  ones 
usually  discussed  are: 

•  cost  risk, 

•  schedule  risk, 

•  technical  risk,  and 

•  production  risk. 
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There  are  numerous  techniques  available  for  evaluating  each  of 
these  types  of  risk.  (The  major  types  of  techniques  to  evaluate 
these  risks  are  discussed  in  Part  IV  of  this  handbook.  ) 

Approaches  have  been  developed  for  quantifying  resource-related 
risks  (cost  and  schedule),  performance-related  technical  risks, 
and  manufacturing/producibility  risks.  However,  there  is  still 
much  to  be  done  in  the  area  of  developing  techniques  for  quanti¬ 
fying  readiness  risks.  Approaches  for  examining  the  collective 
nature  of  materiel  readiness  activities  in  light  of  the  readiness 
goals  of  the  program  need  to  be  developed  so  as’  to  translate 
activity  planning  and  status  into  quantifiable  results.  Until 
these  types  of  tools  are  developed,  the  Program  Manager  must 
largely  rely  on  the  tools  at  his  or  her  disposal  that  deal  with 
the  associated  risks  which  can  influence  readiness  (cost,  sched¬ 
ule,  technical  and  production).  Once  again,  this  is  largely 
attributed  to  the  distributed  nature  of  readiness-related  activities. 

Even  restricting  concern  to  logistics-related  activities, 
does  not  resolve  the  issue.  While  this  will  reduce  the  number  of 
areas  of  concern,  it  does  not  eliminate  the  problem  of  coming  up 
with  a  coordinated  assessment  of  the  logistics-related  risks.  The 
most  appropriate  and  reasonable  approach  the  PM  may  be  able  to 
take  is  to  apply  conventional  risk  analysis  techniques  to  the 
various  readiness-related  activities  and  develop  a  picture  of  the 

risks  i.or  the  group.  There  is  still  much  work  to  be  accomplished 
in  this  area. 

,jr.  r  V, 
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In  addition  to  the  problem  of  managing  for  materiel  readi¬ 
ness,  the  PM  faces  another  set  of  problems  when  applying  concur¬ 
rency  in  the  program. 

CONCURRENCY  AS  THE  PROGRAM  MANAGER'S  DILEMMA 

The  use  of  concurrency  as  a  planning,  management  or  acquisi¬ 
tion  strategy  presents  inherent  difficulties  in  program  manage¬ 
ment.  The  very  nature  of  concurrency  implies  a  certain  amount  of 
complexity  in  that  more  than  one  activity  is  going  on  at  a  time. 
There  are  certain  unavoidable  difficulties  associated  with  most 
applications  of  concurrency. 

One  potential  risk  in  using  concurrency  is  that  of  commit¬ 
ting  and  expending  resources  on  an  ultimately  unproductive 
activity.  In  concurrently  scheduling  sequentially  dependent 
activities,  the  success  of  the  dependent  successor  activity  is 
frequently  jeopardized.  Usually  this  application  of  concurrency, 
involves  the  initiation  of  efforts  in  an  activity  originally 
scheduled  to  start  later,  before  completion  of  activities  produc¬ 
ing  input  to  the  originally  later  activity.  In  doing  this,  the 
manager  makes  certain  assumptions  concerning  the  results  of  the 
predecessor  activity.  Should  those  assumptions  prove  wrong,  the 
efforts  in  the  dependent  activity  may  have  to  be  revised, 
replaced  with  another  approach,  or  possibly  discarded.  The 
resources  spent  on  the  effort  are  at  risk,  and  additional 
resources  may  have  to  be  expended  in  order  to  correct  or  compen-- 
sate.  This,  in  turn,  can  create  a  chain  reaction  of  pressure  on 
the  schedule  in  other  places,  forcing  the  use  of  more  concurrency. 
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Another  difficulty  is  that  communication  and  control  of 
information  becomes  more  difficult.  In  cases  where  normally 
sequential  activities  have  been  scheduled  to  overlap,  information 
on  the  changing  status  of  these  activities  becomes  vital.  Decis¬ 
ions  in  the  predecessor  activity  can  have  significant  (and  some¬ 
times  detrimental)  impacts  on  the  dependent  activity.  Conversely, 
decisions  made  in  the  latter  activity  can  ripple  backwards, 
influencing  the  final  outcome  of  the  predecessor  activity. 

Managers  are  faced  with  the  need  to  have  sufficiently  responsive 
information  control  and  status  monitoring  systems  to  support  the 
more  intensive  management  requirements  that  come  with  using 
concurrency . 


still  another  difficulty  associated  with  using  concurrency 
is  the  substantial  increase  in  risk  to  the  program  that  can 
occur.  Generally,  concurrency  requires  distributing  resources  in 
a  way  different  from  that  of  a  sequentially  dependent  schedule. 

The  manager  may  have  more  than  one  fully  manned  task  occurring  at 
the  same  time,  if  co-development  is  planned.  In  this  case,  the 
personnel  resources  assigned  to  the  task  will  be  at  a  much  higher 
level  than  would  generally  be  required  if  the  tasks  were  not  con¬ 
current.  The  probability  of  success  of  each  approach  will  indi¬ 
cate  how  much  of  these  resources  are  at  risk  of  not  being  successful 
Risks  can  also  be  incurred  in  the  more  conventional  situa¬ 
tion  of  sequentially  dependent  activities  occurring  at  the  same 
time.  Depending  on  the  manloading  of  these  tasks,  the  same 
personnel  could  have  originally  been  planned  to  work  on  both 
tasks,  one  after  the  other.  Concurrency  may  require  them  to  work 
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on  both  tasks  at  the  same  time,  but  with  a  lower  level  of  produc¬ 
tion  on  each.  In  order  to  maintain  a  schedule,  this  may  force 
the  taking  of  shortcuts  (e.g.,  deferring  or  eliminating  documen¬ 
tation),  increasing  risks  that  the  documentation  or  deferred 
analysis  may  not  be  available  when  needed;  or  that  the  concur¬ 
rently  performed  activities  will  not  be  effectively  or  effi¬ 
ciently  performed. 

It  is  important  for  the  manager  to; 

•  understand  the  potential  risks  to  the  program  of 
applying  concurrency;  and 

•  be  able  to  make  judicious  decisions  concerning  how, 
when  and  where  it  is  appropriate  to  use  concurrency. 

He  must  be  able  to  make  trade-offs  between  the  benefits  and  risks 
of  using  a  selected  alternative.  In  addition  to  the  problem  of 
managing  the  concurrency  and  being  able  to  make  effective  trade¬ 
offs,  the  PM  must  also  be  able  to  apply  the  results  of  risk 
analyses  to  resolving  management  problems. 

INTEGRATION  OF  RISK  ANALYSIS  AND  MANAGEMENT 

Program  Managers  are  required  to  perform  risk  analyses  in 
order  to  identify  and  monitor  elements  of  their  programs  which 
may  threaten  their  planned  completion.  Generally  this  involves 
evaluating  the  current  and  projected  status  of  key  indicators  and 
parameters.  Many  techniques  have  been  developed  to  assist  in 
this.  Exhaustive  work  has  been  performed  in  the  area  of  cost, 
schedule,  technical  and  production  risk  assessment.  The  program 
office,  or  more  frequently  the  contractor,  reports  on  the  results 


of  the  various  risk  analyses  conducted  in  conformance  with  direction. 

The  concern  of  the  Program  Manager  is  the  integration  of  the 
results  of  these  analyses  in  the  actual  management  of  the  activi¬ 
ties.  This  raises  the  whole  question  of  the  intention  and  pur¬ 
pose  of  risk  analysis.  Risk  analysis  is  fundamentally  aimed  at 
identifying  those  elements  of  the  program  that  could  jeopardize 
the  ultimate  success  of  the  program.  Rather  than  assume  that 
everything  will  go  right  or  as  planned  (the  Cockeyed  Optimist 
approach)  or  that  the  worst  case  should  be  planned  for  (the 
Murphy's  Law  Will  Prevail  approach),  risk  analysis  is  intended 
to  identify  and  assess  the  magnitude  of  real  and  potential 
dangers.  Ideally,  if  the  risks  are  accurately  identified  and 
effectively  managed,  then  the  failure  should  not  occur.  Con¬ 
versely,  one  can  always  in  retrospect  tell  exactly  where  the  real 
risks  were  hiding  when  the  worst  comes  true.  Risk  analysis  and 
management  is,  then,  a  form  of  crisis  avoidance. 

A  big  question  facing  the  Program  Manager  is  how  best  to  use 
the  results  of  risk  analysis.  Research  has  shown  that  risk 
management  is  frequently  on  a  separate  path  from  risk  analysis. 

The  risk  analysis  in  many  cases  may  be  primarily  used  to  "answer 
the  mail."  Risk  management  may  be  an  implicit  rather  than 
e_xplicit  activity.  This  is  important  to  recognize.  Just  because 
something  is  not  called  risk  management,  it  doesn't  mean  that  the 
risks  are  not  being  managed!  However,  there  should  be  a  clear 
link  between  the  results  of  the  risk  analysis  and  the  management 
of  the  risks  that  have  been  identified  through  the  analysis. 
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Interviews  with  experienced  Government  and  contractor  per¬ 
sonnel  show  that  frequently  risk  analysis  is  conducted  by  a 
separate  group  (the  operations  research  group  in  the  corner). 
Varying  forms  of  discipline  may  be  applied  in  conducting  the  risk 
analysis.  Although  these  techniques  may  be  rigorously  evaluated 
initially,  once  approved  they  are  often  of  little  concern  to  any 

but  the  analysts.  Reports  are  generated  as  required,  the  con- 

I 

tents  may  or  may  not  be  surprising  to  the  managers  receiving 
them.  In  well  run,  effectively  managed  and  structured  programs, 
the  reports  on  the  assessment  of  the  high,  medium  and  low-risk 
elements  may  not  come  as  a  surprise,  since  they  have  usually  been 
identified  by  the  participants.  Risk  analysis  can  assist  them  in 
identifying  alternatives  and  considering  the  magnitude  of  the 
potential  problems.  The  analysis  quantifies,  in  many  cases,  the 
qualitative  evaluation  of  the  participants  as  to  areas  of  con¬ 
cern,  magnitude  of  the  potential  hazard,  viability  of  alternative 
courses  of  action,  and  probability  of  success. 

Frequently,  risks  are  managed  implicitly  throught  the  estab¬ 
lishment  of  the  management  control  systems.  Programs  in  the 
Ballistic  Missle  Office,  as  an  example,  are  organized  to  have 
regularly  scheduled  program  element  progress  reviews.  The  status 
of  the  cost,  schedule,  and  technical  efforts  are  rigorously 
evaluated  and  action  plans  for  managing  concerns  (risks)  are 
developed  and  monitored.  This  is  implicit  or  imbedded  risk 
management  in  that  the  process  fosters  managing  problems  effec¬ 
tively,  however,  it  is  not  explicitly  called  "risk  management". 
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Another  implicit  approach  to  risk  management  advocated  in 
most  of  the  acquisition  and  design  management  guides  is  the 
rigorous  and  thorough  review  and  validation  of  system  require¬ 
ments,  starting  with  program  initiation.  In  this  way,  the  risk 
of  designing  a  system  which  has  unnecessary,  inappropriate  dr 
unrealistic  technical  requirements  can  be  reduced. 

In  managing  risks,  particularly  materiel  readiness  risks,  it 
becomes  a  particular  concern  of  the  managers,  especially  the 
Program  Manager,  to  be  able  to  integrate  the  risk  assessments 
with  management  initiatives. 
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PART  II.  PROGRAM  FACTORS  AND  RELATIONSHIPS 


Chapter  3 . 
Chapter  4 . 
Chapter  5. 


Overview  of  the  System  Acquisition  Process 
Factors  Influencing  Materiel  Readiness 
Concurrency-Related  Factors 


PART  II .  PROGRAM  FACTORS  AND  RELATIONSHIPS 


This  section  focuses  on  the  program  activities  and  elements 
that  relate  to  materiel  readiness .  A  brief  overview  of  major 
activities  that  occur  in  most  Air  Force  acquisitions  and  the 
typical  organizations  in  the  Program  Office  (PO)  is  in 
Chapter  3 .  In  Chapter  4  the  program  activities  that  more 
directly  influence  materiel  readiness  are  discussed  in  greater 
detail.  Chapter  5  considers  how  some  of  these  activities  may  be 
influenced  by  the  application  of  concurrency. 

Appendix  C  provides  additional  details  on  the  key  acquisi¬ 
tion  activities,  as  outlined  in  A  Guide  for  Program  Management, 
AFSC  Pamphlet  800-3,  now  in  revision.  Appendix  D  includes  addi¬ 
tional  information  on  Integrated  Logistics  Support/Logistics 
Support  Analysis . 
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CHAPTER  3 .  OVERVIEW  OF  THE  SYSTEM  ACQUISITION  PROCESS 

•  Major  Air  Force  Acquisition  Activities 

•  Typical  Acquisition  Program  Office  Organization 
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CHAPTER  3.  OVERVIEW  OF  THE  SYSTEM  ACQUISITION  PROCESS 


Systems  acquired  by  the  DoD  are  designed  and  developed 

according  to  a  structured  process  of  phased  activities  and  deci- 

1/ 

sion  milestones.  The  phases  are  designed  to  allow  for  a 
deliberate  and  careful  determination  of  the  requirements  for  a 
new  system,  minimizing  the  risks  of  committing  resources  to  an 
unnecessary,  unproductive,  or  impractical  venture.  The  decision 
milestones  are  intended  to  determine,  at  key  periods  in  the 
development  process,  if  the  system  design  is  progressing  ade¬ 
quately  to  make  it  reasonable  to  continue  the  investment,  or  if 
the  need  and  the  design/development  approach  are  still  appro¬ 
priate  . 

The  process  for  acquiring  systems  within  the  DoD  is  institu¬ 
tionalized  through  a  series  of  regulations.  The  initial  policy 
is  set  by  OSD.  The  regulations,  in  the  form  of  directives  and 
instructions,  apply  to  all  of  the  Services.  The  Services,  in 
turn,  issue  their  own  complementary  regulations  elaborating  on 
the  OSD  policies,  tailoring  the  specific  application  of  the 
policies  to  the  individual  characteristics  of  the  Service.  Each 
major  command,  in  turn,  issues  its  own  implementing  regulations, 
or,  as  in  the  case  of  the  major  acquisition  guidance  from  AFSC, 
in  pamphlets. 

The  major  policy  directives  and  instructions  issued  by  OSD 
that  relate  to  the  acquisition  process  are  in  the  5000  series. 

V  The  term  "system,"  as  used  in  this  guide,  refers  to  not  only  the 
primary  mission  hardware  and  software,  but  also  the  support 
equipment,  technical  data,  personnel,  etc. 
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While  there  are  numerous  directives  and  instructions  that  are 
specifically  germane  to  managing  systems  acquistions,  the  key¬ 
stone  policies  are  set  down  in  two  documents: 

•  DoD  Directive  5000.1,  Major  System  Acquisitions,  and 

•  DoD  Instruction  5000.2,  Major  System  Acauisition 

Procedures .  - 

These  regulations  are  reviewed  and  revised  as  policy  is  changed, 

as  are  the  Service  implementing  regulations,  instructions,  and 
2/ 

pamphlets.  Included  in  these  policies  are  descriptions  of  the 
roles  and  responsibilities  of  organizations  involved  in  systems 
acquisition,  and  the  analyses  which  must  be  performed  in  each 
stage  of  the  system's  development. 

In  the  following  discussion  the  activities  in  the  various 
phases  of  the  systems  acquisition  process  will  be  only  briefly 
discussed.  For  more  in-depth  descriptions  there  are  numerous 
guides  published  by  the  Defense  Systems  Management  College  (DSMC) 
and  various  groups  within  the  Air  Force  and  the  other  Services. 
Many  of  these  are  referenced  in  Appendix  A.  In  this  chapter, 
five  major  sources  have  been  used: 


The  most  recent  versions  of  these  regulations  were  issued  on  29 
March  1982  ( DoDD  5000.1)  and  8  March  1983  (DoDI  5000.2).  Many  of 
the  handbooks,  guidebooks  and  Service  regulations  referenced  in 
this  handbook  do  not  reflect  these  most  recent  revisions,  partic- 
ularly  relating  to  milestone  decision  authorities.  Where 
possible,  differences  between  the  former  process  as  represented 
in  these  references,  and  the  current  process  have  been  noted  or 
corrected.  However,  the  reader  should  be  aware  that  in  all  cases 
that  has  not  been  possible  and  should  review  the  current  versions 
of  the  pertinent  OSD  and  Air  Force  regulations  for  specific  dis¬ 
cussions  on  current  responsibilities.  Many  of  the  relevant  Air 
Force  regulations  are  being  revised  but  are  substantially  ade¬ 
quate  for  the  purposes  of  this  handbook.  Listings  of  pertinent 
DoD  regulations,  organized  by  topic,  are  contained  in  Appendix  A. 
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•  Huffman,  Major  James  W. ,  Lozito,  Major  Vincent  J.,  Jr. 
and  Snyder,  Major  Larry  A.,  Weapon  Systems  Acquisition 
Guide ,  Air  Command  and  Staff  College,  Air"  University, 
Maxwell  AFB,  Alabama,  May  1980. 

•  Runkle,  Captain  Marty  T.  and  Smith,  Captain  Michael  L., 
Systems  Acquisition  Guide,  Air  Command  and  Staff 
College,  Air  University,  Maxwell  AFB,  Alabama,  May 
1978. 

•  Systems  Engineering  Management  Guide,  Defense  Systems 
Management  College,  Ft.  Belvoir,  Virginia,  3  October 
1983. 

•  Navy  Program  Manager's  Guide,  NAVMAT  P-9494,  Naval 
Material  Command,  July  1983. 

•  A  Guide  for  Program  Management,  AFSCP  800-3,  Air  Force 
Systems  Command!,  9  April  1976  (in  revision). 


MAJOR  AIR  FORCE  ACQUISITION  ACTIVITIES 


The  system  acquisition  process  can  be  thought  of  as  having 
six  phases : 

•  the  Mission  Area  Analysis, 

•  the  Concept  Exploration  (CE)  Phase, 

•  the  Demonstration  and  Validation  (D&V)  Phase, 

•  the  Full  Scale  Development  (FSD)  Phase, 

•  the  Production  Phase,  and 

•  the  Deployment  Phase. 

This  handbook  deals  primarily  with  those  activities  in  the  four 
middle  phases  which  can  influence  the  sixth  phase. 

Exhibit  3-1  summarizes  the  acquisition  life  cycle  technical 
activities  in  the  major  program  areas  for  a  typical  major  system 
acquisition,  from  program  initiation  to  disposal.  As  can  be  seen, 
activities  occur  in  all  areas,  with  certain  activities  being 


i 
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Exhibit  3-1.  SUMMARY  OF  ACQUISITION  TECHNICAL  ACTIVITIES 
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repeated  within  each  phase,  specifically  those  related  to  mile¬ 
stone  decisions.  Each  of  the  activities  shown  on  this  exhibit 
represents  a  set  of  tasks  required  to  support  the  activity, 
marked  by  initiation  and  completion  milestones,  or  events.  As 
can  be  seen  in  this  exhibit,  activities  in  each  of  the  program 
areas  progress  through  each  phase  toward  a  sufficiently  complete 
stage  for  "the  concurrent  delivery  of  system  and  all  requisite 
items  of  support."  These  activities  are  only  generally  discussed 
in  this  chapter.  The  activities  most  directly  related  to  materiel 
readiness  are  discussed  in  Chapter  4. 

Exhibit  3-2  gives  a  summary  of  the  acquisition  process  from 
the  perspective  of  the  major  phase  activities  relating  to  system 
design  and  testing.  The  major  emphasis  in  this  handbook  is  on 
the  activities  in  the  Concept  Exploration,  Demonstration  and 
Validation,  and  Full  Scale  Production  Phases.  These  are  the 
phases  when  design  and  management  decisions  relating  to  materiel 
readiness  may  be  jeopardized  by  the  imposition  of  concurrency. 
While  the  overlapping  of  the  FSD  and  Production  Phases  is  when 
concurrency  relating  to  materiel  readiness  can  nave  a  most  signi¬ 
ficant  impact,  the  decisions  on  how  to  manage  uhe  risks  will  be 
made  primarily  in  FSD.  Where  appropriate,  however.  Production 
Phase  activities  will  be  discussed. 

The  Mission  Area  Analysis  determines  the  initial  requirement 
for  a  new  system,  or  the  need  to  modify  an  existing  system.  Based 
on  this  analysis,  a  number  of  documents  are  produced  including 
the  two  key  ones  discussed  below: 
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•  the  Justification  for  Major  System  New  Start  (JMSNS), 
the  statement  of  the  circumstances  for  requiring  the 
new  or  modified  system;  and 

•  the  Program  Decision  Memorandum  ( PDM ) ,  the  authoriza¬ 
tion  to  initiate  the  program. 

The  SECDEF  approval  of  the  Program  Objective  Memorandum 
(POM),  of  which  the  JMSNS  is  part,  and  the  signing  of  the  PDM 
initiates  the  first  of  the  four  developmental  phases:  Concept 
Exp lor  a t i on .  These  phases  involve  sequences  of  activities  per¬ 
formed  by  various  organizations  in  specific  functional  areas.  An 
example  of  such  a  set  of  activities  is  shown  in  Exhibits  3-3A  and 
3-3B.  Shown  are  the  major  activities  involving  the  Air  Force  and 
the  contractor  in  performing  the  Systems  Engineering  and  Config¬ 
uration  Management  functions. 

The  focus  of  each  phase  is  to  develop  a  more  refined 
description  of  the  system  to  be  developed.  These  descriptions  of 
requirements  are  called  baselines  and  the  specifications  relating 
to  the  baseline  are  the  major  focus  of  the  Systems  Engineering/ 
Configuration  Management  Functions.  Most  of  the  design-related, 
versus  management-related,  activities  are  focused  toward  develop¬ 
ing  the  baselines  and  specifications.  This  includes  not  only  the 
hardware  specifications,  but  also  baseline  descriptions  of  the 
software.  (These  are  discussed  in  greater  detail  in  Chapter  4.) 

The  Concept  Exploration  Phase  activities  concentrate  on 
exploring  the  various  concepts  open  to  the  Air  Force  to  fulfill 
the  need  stated  in  the  JMSNS.  These  activities  include  the 
following : 

•  validate  the  Required  Operational  Capability  (ROC); 
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Exhibit  3-3B.  INTERRELATIONSHIPS  BETWEEN  SYSTEMS  ENGINEERING 
AND  CONFIGURATION  MANAGEMENT:  FULL  SCALE  DEVELOPMENT 

AND  PRODUCTION  PHASE 


study  alternative  design  concepts; 

develop  preliminary  systems  specifications; 

develop  a  proposed  statement  of  work  (SON)  for  the  con¬ 
tracts  to  be  let  for  the  following  development  phase; 

prepare  the  System  Concept  Paper  (SCP),  describing: 

initial  program  alternatives  based  on  initial 
studies  and  analyses  of  design  concepts, 

alternative  acquisition  strategies, 

expected  operational  capabilities, 

industrial  base  capacity, 

readiness,  support  and  personnel  requirements,  and 
-  cost  estimates; 
define  life  cycle  costs; 

prepare  the  Test  and  Evaluation  Master  Plan  (TEMP); 

determine  computer  resource  policy  concerning  Hiqh 
Order  Language  and  standardization; 

develop  ILS  strategy; 

investigate  alternative  support  and  maintenance 
concepts ; 

evaluate  production  feasibility; 

assess  cost,  schedule,  technical  and  production  risks 
of  alternatives; 

establish  design  to  cost/manufacturing  cost  goals; 
develop  acquisition  and  manufacturing  strategies; 
develop  ILS  master  plan;  and 
•  begin  logistic  support  analysis. 

Exhibit  3-4  lists  the  generic  inputs,  principal  activities 
and  outputs  of  the  Concept  Exploration  Phase.  While  these  may  be 
somewhat  different  for  individual  programs  and  less  than  major 


3-10 


4 


INPUTS 


•  APPRO VEO  JMSN  S/OR/ROC 

INCLUDED  IN  POM 

•  APPROVEO  EXCEPTIONS 

TO  A  FULL  A-109 
ACQUISITION  POLICY 

•TECH  BASE 
OEVELOPERS 

•  POSSIBLE 

•  "ENTREPRENEURS'* 

•  ADVOCATES 

•  AOV  DEVELOPMENT 

PROJECT  OFFICE 

•  BUOGET  LINE  ITEM 


CONCEPT 

exploration 

PHASE 


PRINCIPAL  FUNCTIONS  AND  ACTIVITIES 


A  pM  APPOINTED 
A  PM  CHARTER 


ACQUISITION  STRATEGY 


DEVELOP  APPROVED 


REFINED 


FUNCTIONAL  IMPLEMENT  PLANS 


•PO,  IN-HOUSE  SUPPORT  TEAM  DEVELOP 


CONTRACTING  PROCESS/PREP 


SOLICITATION,  CRITERIA.  EVALUATION 


E 

X 

P 

L 

O  0 
R  F 
A 
T 

I 

0 

N 


ran.  a  -contracts 

D&F  “  AWARDED 


C 

0 

N 

C 

E 

P 

T 

S 


CONTRACT  MONITORING.  REVIEW 


FOLLOW-ON  CONTRACT 


NEGOTIATIONS 


INDEP,  IN-HOUSE  LCC  ESTIMATES 


INDEPENDENT  IN-HOUSE  TECHNICAL, 
SAFETY,  MANNING,  SUPPORTABlLlTY  STUDIES 


DRAFT  TEMP  (WITH  AFOTEC) 


PREPARE  MRD 


REVIEW  PROCESS 


MILESTONE  I  A 


•  TEMP 


•PROPOSEO  FOLLOW-ON 
CONTRACTS)  FOR 
OAV  PHASE 

•  POSSIBLE  SOURCE 

SELECTION  OF 
CONTRACTORS  FOR 
OAV  PHASE 

•  ACQUISITION 

STRATEGY/FUNCTIONAL 
IMPLEMENTATION  PLANS 

•  MLESTONE  I  MRD 
(SCP) 

•  MILESTONE I  OECISION 

MEMORANOUM  <S00M/ 
OTHER) 

•  FUNCTIONAL  BASELINE 


04V  -DEMONSTRATION  ANO  VALIDATION 

OAF  -DETERMINATION  ANO  FINOING 

JUS  NS  -  JUSTIFICATION  FOR  MAJOR  SYSTEM  NEW  START 

MNO  -MISSION  NEEO  DETERMINATION 

MRO  -  MILESTONE  REVIEW  DOCUMENTATION 


OR  -OPERATIONAL  REQUIREMENT 

PM  -PROGRAM  MANAGER 

RAN  -REOUEST  FOR  AUTHORITY  TO  NEGOTIATE 

ROC  -REQUIREO  OPERATIONAL  CAPABILITY 

ICP  -SYSTEM  CONCEPT  PAPER 

SPO  -  SYSTEM  PROGRAM  OFFICE 


Source:  Navy  Program  Manager's  Guide,  P-9494,  Naval  Material 

Command ,  July  1983 


Exhibit  3-4.  INPUTS,  PRINCIPAL  ACTIVITIES,  AND  OUTPUTS 
OF  CONCEPT  EXPLORATION  PHASE 
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systems,  they  tend  to  be  applicable  for  most  large  programs. 

Many  of  the  CE  Phase  activities  involve  planning,  organization 
and  exploration.  Exhibit  3-5  is  the  flow  of  activities  leading 
from  the  ROC  to  the  approved  DCP,  as  illustrated  in  A  Guide  for 
Program  Management  (AFSCP  800-3).  while  some  of  these  activities 
may  have  changed  since  this  pamphlet  was  published  in  1976,  the 
process  used  today  is  functionally  similar.  Appendix  C  contains 
an  annotated  discussion  of  the  block  flow  of  activities  in  this 
exhibit  as  well  as  the  ones  for  the  remaining  phases. 

Neither  Exhibit  3-4  nor  3-5  represents  the  activities  that 
occur  within  the  Program  Office  (PO)  in  detail.  This  is  primar¬ 
ily  due  to  a  fundamental  difficulty  in  representing  the  "major 
tivities  in  what  is  usually  a  very  complex  process.  Exhibit 
3-6  is  an  example  of  a  Work  Breakdown  Structure  (WBS)  for  a  mis¬ 
sile  system.  Program  activities  must  be  organized  to  perform  the 
planning,  design,  review,  monitoring,  and  other  functions,  for 
each  of  the  elements  in  the  WBS.  Below  Level  3  are  still  more 
levels  of  detail,  with  most  large  systems  having  at  least  six 
levels  of  detail.  Program  activities  are  frequently  tied 
directly  to  WBS  elements  for  the  purposes  of  review,  monitoring, 
cost  and  schedule  accounting.  A  preliminary  WBS  is  drafted  as 
soon  as  possible  in  the  program,  usually  using  generic  structures 

such  as  those  found  in  Major  System  Work  Breakdown  Structure, 
MIL-STD-881A. 

ThS  PgS°"stration  and  Validation  ( D&V )  Phase  usually  follows 
the  Concept  Exploration  Phase.  This  phase  is  initiated  by  the 
SECDEF/SAF  Milestone  I  decision  on  program  refinement.  In  this 
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Source:  A  Guide  for  Program  Management/  AFSCP  800-3,  Air  Force  Systems  Command, 

9  April  1976 

Exhibit  3-5.  ACTIVITY  FLOW  IN  CONCEPT  EXPLORATION  PHASE 
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Source:  Navy  Program  Manager's  Guide,  P-9494,  Naval  Material  Command,  July  1983 
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Exhibit  3-6.  THREE-LEVEL  WORK  BREAKDOWN  STRUCTURE 
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phase  the  major  design  effort  defining  the  specific  system  char¬ 
acteristics  takes  place.  The  following  are  the  major  activities 
taking  place: 

•  review  and  evaluate  proposal,  and  select  one  or  more 
contractors ; 

•  review  system  requirements; 

•  perform  trade  studies; 

•  develop  system  definition; 

•  determine  interface  specifications  and  agreements; 

•  perform  system/subsystem  engineering  and  integration; 

•  perform  specification  updating  and  analysis; 

•  define  configuration  control  procedures; 

•  establish  drawings  release  control; 

•  determine  long  leadtime  items; 

•  assess  production  capability; 

•  establish  laboratory  relationships; 

•  update  TEMP  and  determine  test  support  requirements; 

•  conduct  development  test  and  evaluation  prototype  test; 

•  prepare  program  introduction  document  (PID); 

•  establish  management  information  system; 

•  update  life  cycle  cost  estimates; 

•  update  risk  analyses; 

•  prepare  DSARC  II,  Decision  Coordinating  Paper  (DCP)  and 
Integrated  Program  Summary  (IPS); 

•  conduct  Preliminary  Design  Review  ( PDR ) ; 

•  select  computer  program  configuration  items  (CPCI  )  ; 

0  update  Computer  Resources  Integrated  Support  Plan 

(CRISP); 
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•  conduct  software  requirements  verification  and  valida¬ 
tion; 

•  develop  preliminary  manufacturing  plan;  and 

•  develop  contractual  vehicles  for  FSD  contracts. 

Exhibit  3-7  shows  the  major  inputs,  activities  and  outputs 

of  the  Demonstration  and  Validation  Phase.  Many  of  these  are 
continuations  of  activities  begun  in  the  Concept  Exploration 
Phase.  The  general  flow  of  activities  and  responsibilities  in 
this  phase  of  a  system  acquisition  among  OSD,  SAF ,  HQ  USAF,  HQ 
AFSC  and  the  Product  Division  Program  Office  is  discussed  in 

Appendix  C.  Activities  relating  to  materiel  readiness  are  dis¬ 
cussed  in  Chapter  4. 

Following  the  Demonstration  and  Validation  Phase  is  the  Full 
Sc_ale  Development  (FSD)  Phase.  The  major  thrust  of  this  phase  is 
to  convert  the  system  and  subsystem  specifications  developed  in 
the  D&V  Phase  into  detailed  specifications,  converting  the  allo¬ 
cated  baseline  into  a  production  ready  baseline,  and  producing 
full  size  operational  versions  of  the  system  including  pilot 
productions.  The  planning,  design  and  development  tasks  initi¬ 
ated  and  performed  in  the  preceeding  phase  are  largely  culminated 
in  the  FSD  Phase.  Emphasis  shifts  toward  ensuring  a  producable, 
operational  system  with  adequate  field  support  after  deployment. 
The  major  activities  in  the  FSD  Phase  are  summarized  below,  and 
illustrated  in  Exhibit  3-8. 

The  major  activities  in  the  FSD  Phase  focus  on  the  following: 
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Exhibit  3-7.  INPUTS ,  PRINCIPAL  ACTIVITIES  AND  OUTPUTS  OF. 
DEMONSTRATION  AND  VALIDATION  PHASE 
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Exhibit  3-8. 


INPUTS,  PRINCIPAL  ACTIVITIES  AND  OUTPUTS  OF 
FULL-SCALE  DEVELOPMENT  PHASE 
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update  the  Program  Management  Directive  (PMD)  and 
Procurement  Authorization  (PA); 

update  program  documentation? 

review  and  evaluate  proposals  and  select  one  or  more 
contractors  for  FSD  contracts; 

perform  production  readiness  review  (PRR); 

design  complete  logistic  support  system; 

ensure  that  ILS  is  part  of  deslign  trade  offs; 

design  support  equipment; 

update  ILSP ; 

collect  LSA  data  and  maintain  LSAR ; 
update  TEMP; 

perform  engineering  development  testing  in  Developmen¬ 
tal  Test  and  Evaluation  (DT&E)  and  Operational  Test  and 
Evaluation  (OT&E); 

finalize  manufacturing  plan; 

update  life  cycle  cost  estimates 

reassess  risks; 

review  design  to  cost  goals; 

revalidate  technical  requirements; 

develop  Type  B  specifications; 

complete  software  design; 

complete  software  coding; 

complete  software  testing; 

develop  contractual  vehicle  for  Production  Contracts; 
continue  configuration  management; 

develop  and  finalize  technical  documentation  (orders 
and  manuals ) ; 

plan  deployment; 

conduct  Critical  Design  Review  ( CDR ) ; 
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•  conduct  Design  Certification  Review  (DCR);  and 

•  determine  if  system  is  appropriate  for  transitioning  to 
production  -  Milestone  III  SAF  decision. 

The  final  acquisition  phase  is  the  Production  or  Production/ 
--ePloyment  Phase'  Ir>  this  phase  the  system  is  actually  produced 
m  quantity  and  deployed.  This  phase  can  extend  over  several 
years.  The  activities  in  this  phase  are  oriented  towared  trans¬ 
ferring  program  management  responsibilities  from  AFSC  to  AFLC ; 
implementing  the  logistics  support  system;  correcting  the  design 
based  on  late  test  results  and  field  data;  and  implementing  pre¬ 
planned  product  improvements.  The  major  activities  are  discussed 
below.  Exhibit  3-9  lists  the  major  inputs,  activities  and  out¬ 
puts  of  this  phase. 

The  major  activities  in  the  Production  Phase  are  the 
f ol lowing : 


•  update  the  acquisition  and  manufacturing  plans  and 
strateg ies ; 

•  select  one  or  more  production  contractors; 

•  reevaluate  IOC  target; 

•  initiate  field  support; 

•  implement  deployment  plans  and  schedule; 

•  produce  and  finalize  production  documentation  package; 

•  produce  and  firialize  current  software  package; 

•  finalize  technical  documentation; 

•  train  initial  cadre  of  personnel; 

•  validate  training  system; 

•  conduct  Physical  Configuration  Audit  (PCA); 
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Source:  Navy  Program  Manager's  Guide,  P-9494,  Naval  Material 

Command,  July  1983 


Exhibit  3-9.  INPUTS,  PRINCIPAL  ACTIVITIES  AND  OUTPUTS  OF 
PRODUCTION  AND  DEPLOYMENT  PHASE 


3-21 


► 


# 


conduct  final  operational  test  and  evaluation; 
continue  implementation  of  change  orders; 
implement  p3i  program; 

verify  and  maintain  system  design  and  production  speci¬ 
fications; 

deliver  system  and  support  system; 

transfer  program  management  responsibility  from  AFSC  to 
AFLC;  and 


•  develop  second  production  source,  as  required. 

This  has  been  intended  as  a  very  cursory  review  of  the  major 
system  acquisition  phase  activities.  The  reader  should  consult 
the  various  regulations  and  guides  referenced  here  and  in 
Appendix  A  for  detailed  discussions  on  these  activities.  Addi¬ 
tional  detail  regarding  these  activities  is  also  given  in 
Appendix  C. 

TYPICAL  ACQUISITION  PROGRAM  OFFICE  ORGANIZATION 

In  addition  to  the  acquisition  activities,  there  is  another 
side  to  the  acquisition  process:  the  program  office  ( PO ) .  These 
are  the  people  who  perform  the  acquisition  activities.  The 
program  office  can  be  thought  of  including  not  only  the  Air  Force 
personnel  in  an  AFSC  Product  Division,  but  also  the  civilian 
contractor  performing  the  design,  development  and  production 
responsibilities. 

In  this  part  of  the  chapter  the  basic  structure  of  a 
"typical”  program  office  is  briefly  discussed.  As  with  the  pre- 
ceeding  overview  of  acquisition  activities,  only  a  very  summary 
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consideration  of  the  organizations  generally  found  in  a  PO  is 
given  here.  This  is  for  two  reasons: 

•  There  are  numerous  guides  and  studies  available  which 
discuss  PO  organization  and  responsibilities. 

•  The  size  and  structure  of  the  PO  must  be  tailored  to 
the  specific  program. 

This  part  of  the  chapter  is  primarily  intended  to  give  the  reader 
an  orientation  in  order  to  understand  the  discussions  of  materiel 
readiness-related  activities  in  Chapter  4. 

Exhibit  3—10  gives  an  indication  of  the  various  groups  with 
which  the  PO  must  interact.  The  PO  is  usually  divided  into  a 
number  of  functional  areas  which  may  be  called  Directorates, 
Divisions,  or  Branches.  These  groups  are  responsible  for  manag¬ 
ing  and  performing  the  tasks  identified  in  the  various  plans. 

There  are  two  basic  types  of  POs:  the  single  program  PO 
which  may  be  organized  to  acquire  a  major  or  super  program  (e.g., 
F-16,  B-1B,  Peacekeeper  programs);  and  the  "basket"  PO  which  is 
organized  to  develop  and  acquire  multiple  systems  of  the  same 
type.  Exhibits  3-11  and  3-12  illustrate  the  structural  differ¬ 
ences  in  PO  organization  between  a  single  program  and  multiple 
program  PO.  As  can  be  seen,  the  same  functions  are  shown  for 
each  type  of  PO,  however,  some  functions  are  shared  among  the 
programs  in  the  multiple  program  PO,  while  others  are  duplicated 
for  each  program.  Exhibits  3—13  and  3—14  show  examples  of  vari¬ 
ations  on  the  single  program  PO  structure.  Exhibit  3-13  depicts 
the  Ballistic  Missile  Office  Structure  and  Exhibit  3-14  illus¬ 
trates  the  general  structure  of  the  F-16  PO.  Both  illustrate  the 
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Source:  Weapon  System  Acquisition  Guide,  (Air  Command  and  Staff  College,  May  1981) 


Exhibit  3-10.  PROGRAM  OFFICE  RELATIONSHIPS 


Source:  Weapon  System  Acquisition  Guide,  (Air  Command  and  Staff  College,  May  1981) 


Exhibit  3-11.  TYPICAL  SINGLE  PROGRAM  PO 
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Source:  Weapon  System  Acquisition  Guide,  (Air  Command  and  Staff  College,  May  1981) 
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Exhibit  3-12.  TYPICAL  MULTIPLE  PROGRAM  PO 
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Source:  Engineering  Project  Officers1  Manual,  Ballistic  Missile  Office,  May  1983 


Exhibit  3-13 .  BALLISTIC  MISSILE  OFFICE  ORGANIZATION 
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Exhibit  3-14.  F-16  SPO  ORGANIZATION  CHART 
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organizational  tailoring  of  PO  organizations  to  fit  the  needs  of 
the  program.  The  BMO  organization  includes  many  groups  not 
usually  separately  identified  in  a  PO.  The  multinational 
nature  of  the  F-16  program  is  indicated  by  the  Multinational 
Programs  Directorate  and  International  Finance  Division. 

1.  Systems  Engineering  Directorate^ 

The  Systems  Engineering  Directorate  is  the  organization 
within  the  PO  primarily  responsible  for  the  systems  engineering 
and  technical  design  for  the  system.  This  directorate  manages 
the  in-house  engineering  activities  and  the  contractor  engineer¬ 
ing  activities  including  system  and  subsystem  integation  and 
specialty  engineering. 

The  Systems  Engineering  Directorate  is  also  responsible 

for : 

•  conducting  system  analysis  and  trade  off  studies; 

•  approving  performance  specifications; 

•  assisting  in  technical  order  validation  and  verifica¬ 

tion; 

•  conducting  technical  reviews; 

•  assisting  in  test  and  evaluation  efforts,  particularly 

developmental  test  and  evaluation;  and 

•  participating  in  the  Test  Planning  Working  Group 

( TPWG ) ,  Computer  Resources  Working  Group  (CRWG)  and 
Interface  Control  Working  Group  (ICWG). 

The  overall  engineering  management  task  involves  defining  system 

performance  parameters  and  configurations;  planning  control  of 

— /  The  major  portion  of  this  discussion  has  been  adapted  from  the 
Weapon  System  Acquisition  Guide,  (Huffman,  Lozito  and  Snyder,  Air 
Command  and  Staff  College,  May  1981). 
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technical  program  tasks;  integration  of  the  results  of  the 

specialty  engineering  efforts  in  the  total  design;  and  optimizing 

cost,  schedule,  readiness  and  technical  performance  considera- 
— / 

tions .  An  example  of  the  organization  of  an  Engineering 
Directorate  for  an  aircraft  pregram  is  shown  in  Appendix  C. 

2*  Configuration  Management  Directorate"7 

The  Configuration  Management  Directorate  is  responsible 
for  overseeing  the  control  of  the  hardware  and  software  configu¬ 
ration  throughout  the  acquisition  process.  This  involves; 

•  identifying  the  functional  and  physical  characteristics 
of  selected  system  components  (Configuration  Items), 

*  maintaining  the  data  files  on  the  status  of  the  Cl 
characteristics; 


controlling  the  changes  to  these  items;  and 
performing  configuration  audits. 

The  primary  vehicles  for  tracking  the  conf igrations  are  through 
the  functional,  allocated  and  product  baselines,  and  the  Type  A, 

B,  C,  and  D  specifications.  This  directorate  is  also  responsible 
for  overseeing  technical  publications  and  data  management.-7  The 

organization  of  a  typical  Configuration  Management  Directorate  is 
shown  in  Appendix  C. 

^°y^ddltl°nal  information  in  engineering  management  responsibil¬ 
ities  see  A  Guide  for  Program  Management.  AFSCP  800-3,  and  System 
Engineering  Management  Guide,  DSMC/  3  October-  1983.  - 

i/  The  substance  of  this  discussion  is  taken  from  the  Systems 
Requisition  Guide,  (Runkle  and  Smith,  Air  Command  and  Staff 
College,  May  1978.) 

~  »nn  ®dd?;ti°?al  information  on  configuration  management  see  AFSCP 
„00-7,  Configuration  Management,  and  the  System  Engineering 
Management  Guide.  - =• 


3-30 


3 .  Program  Control  Directorate"1^ 

The  Program  Control  Directorate  is  responsible  for: 

•  program  planning  and  programming; 

•  progress  tracking/schedule  monitoring; 

•  maintenance  of  the  master  schedules; 

•  developing  the  program  budget  submissions  for  the  PPBS 
cycle; 

•  analyzing  the  financial  status  of  the  program; 

•  coordinating  risk  analysis  results  in  the  program 
acquisition  strategy;  and 

•  developing  program  forecasts  of  risks. 

The  Program  Control  Directorate  usually  includes  the  financial 
management  functions,  the  personnel  and  resource  planning  groups, 
liaison  offices  for  external  groups  (Congress,  visiting  dignitar¬ 
ies,  analysts  from  other  organizations,  etc.)  They  maintain 
oversight  of  the  contractor's  cost/schedule  control  system 
criteria  (C/SCSC)  and  analyze  the  reports  the  contractor  produce 
in  response  to  this  requirement.  An  illustration  of  the  organi¬ 
zation  of  a  typical  Program  Control  Directorate  is  given  in  Appendix  C. 

4 .  Contracting  Directorate 

The  Contracting  Directorate  is  responsible  for  conduct¬ 
ing  the  activities  associated  with  developing  the  contractual 
vehicles  for  acquiring  contractor  services,  supplies,  equipment, 
support ,  etc.;  participating  in  the  selection  of  sources,  nego— 
tiating  contracts  and  administering  modifications  to  contracts. 
Responsibilities  of  the  Contractoring  Directorate  include: 

1/  The  substance  of  this  discussion  is  taken  from  the  Weapon  System 
Acquisition  Guide.  - c - * - 
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developing  the  Request  for  Proposal  (RFP),  invitation 
for  Bids  (IFB)  and  Request  for  Quotes  ( RFQ ) ; 

developing  the  determination  and  findings  ( D&F ) ; 

determining  the  type  of  contract  to  be  let; 

determining  the  schedule  and  terms  for  the  contract; 

deciding  if  the  contract  should  be  competed; 

issuing  the  RFP,  IFB  or  RFQ; 

receiving  the  proposals  in  response; 

participating  in  the  source  selection  process; 

chairing  the  negotiation  team  for  negotiating  the  final 
contract  terms; 

approving  and  executing  the  signed  contract; 

working  with  the  Air  Force  Plant  Representative  Office 
(AFPRO)  to  ensure  that  the  contractor  fulfills  the 
terms  of  the  contract; 

•  negotiating  and  adminstrating  any  contract  modifica¬ 
tions;  and 

•  evaluating  contract  impacts  of  Engineering  Change 
Proposals. 

The  organization  of  a  typical  Contracting  Division  is  given  in 
Appendix  C. 


5 .  Manufacturing  Management  Directorate^ 

The  Manufacturing  Management  Directorate  is  responsible 

r 

for  overseeing  all  manufacturing  activities,  including  developing 
the  manufacturing  plan  and  strategies,  producibililty  analysis, 
production  planning  and  control,  production  demonstration  and 
testing,  manufacturing  method  development,  and  overseeing  of 


— ^  The  substance  of  this  discussion  is  based  on  information  con¬ 
tained  in  Department  of  Defense  Manufacturing  Management  Handbook 
for  Program  Managers,  first  edition,  DSMC ,  January  1982. 
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fabrication,  assembly,  installation,  checkout,  scheduling  and 
program  surveillance. 

The  manufacturing  management  process  has  three  major 
parts:  planning,  analysis,  and  control.  The  planning  phase  con¬ 

siders  factors  such  as: 

•  materiel  acquisition; 

i 

•  adequacy  of  the  labor  force; 

•  engineering  design; 

•  provisions  for  sub-contractor  support; 

•  production  feasibility; 

•  producibi lity  of  the  engineering  design; 

•  new  manufacturing  processes,  facilities,  tools  and  test 
equipment;  and 

•  cost  control  during  design. 

The  analysis  phase  focuses  on  the  success  of  the  manufactur¬ 
ing  process,  its  efficiency,  economy  and  success  in  complying 
with  the  manufacturing  plan.  In  the  design  process  this  activity 
focuses  on  analyzing  the  risks  of  producing  a  given  design.  The 
analysis  aspect  of  manufacturing  management  also  focuses  on  coor¬ 
dinating  manufacturing  impacts  of  decisions  made  by  the  Systems 
Engineering,  Configuration  Management,  and  Program  Control  Direc¬ 
torates  . 

The  most  important  aspect  of  manufacturing  management  is  the 

conduct  of  the  Production  Readiness  Review  ( PRR ) . 

"The  PRR  is  a  formal  inspection  conducted  by 
the  Government  ( PO  with  APPRO  support)  to 
verify  that  the  production  design,  planning, 
and  associated  preparations  for  a  system  have 
progressed  to  the  point  where  a  production 
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comm  it  merit  can  be  made  without  incurring 
unacceptable  risks  of  breaching  thresholds  of 
schedule,  performance,  cost,  or  other  estab¬ 
lished  criteria. "9/ 

The  PRR  effort  spans  the  FSD  Phase  and  focuses  on  the  manufac¬ 
turers  planning,  equipment,  resources,  methods,  etc. 

The  final  manufacturing  management  phase  is  control,  which 

focuses  on  monitoring  the  manufacturing  status  through  contractor 

10/ 

status  reports.  The  basic  organization  of  a  Manufacturing 

Management  Directorate  is  shown  in  Appendix  C. 

6 .  Test  and  Evaluation  Directorate"""'^ 

The  Test  and  Evaluation  Directorate  is  responsible  for 
the  overall  coordination  of  the  system  test  and  evaluation 
effort.  The  main  purpose  of  the  T&E  effort  is  to: 

•  assess  and  reduce  risks; 

•  evaluate  the  system's  operational  effectiveness  and 

suitability;  and 

t 

•  identify  system  deficiencies. 

Test  and  Evaluation  Directorate  activities  begin  early  in 
the  CE  Phase  with  the  development  of  the  Test  and  Evaluation 
Master  Plan.  They  continue  through  development,  initial  and 
final  operational  test  and  evaluation.  In  the  Concept  Explora¬ 
tion  Phase,  T&E  efforts  are  focused  on  assisting  in  the  selection 


Based  on  information  contained  in 
DoDI  5000.38,  24  January  1979. 


Production  Readiness  Reviews, 


12/For  more  information  see  also  Manufacturing  Management  for  Air 
Force  Acgu  is  it  ions  ,  AFR  800-9,  "'l  October'  1979" - - 

iL/This  discussion  is  based  on  information  in  the  Weapon  System 
Acguisition  Guide.  - c - - 


3-34 


D&V  Phase  test  and  evaluation  activi- 


■4 


of  alternative  concepts, 
ties  involve  efforts  to  minimize  design  risks;  demonstrate  tech¬ 
nical  and  operational  feasibility;  and  prototype  testing.  FSD 
Phase  efforts  are  oriented  toward  demonstration  that  the  engineer¬ 
ing  design  is  complete;  demonstration  that  the  terms  of  the  con¬ 
tract  have  been  complied  with;  identification  and  correction  of 
deficiences;  and  estimation  of  operational  suitability  and  effec¬ 
tiveness.  Production  and  Deployment  test  and  evaluation  activi¬ 
ties  are  intended  to  evaluate  system  improvements ,  corrections 
and  modifications;  refine  estimates  of  operational  effectiveness 
and  suitability;  and  evaluate  the  system  under  changing  environments. 

A  typical  Test  and  Evaluation  Directorate  organization  for  a 
multiple  program  PO  is  shown  in  Appendix  C.  For  a  single  program 
PO,  the  T&E  Directorate  may  also  be  responsible  for  deployment, 
in  which  case  it  may  be  organized  similarly  to  the  F-16  Program 
Directorate  of  Deployment  and  Test.  This  directorate  has  three 
divisions : 


•  Field  Support  and  Data, 

^  Deployment,  and 

•  Test  Support. 

Test  and  Evaluation  activities  will  be  discussed  in  more  detail 
in  Chapter  4. 

_  12  / 

7 .  Integrated  Logistics  Support  Directorate — ' 

The  Integrated  Logistics  Support  Directorate  is  the 

last  of  the  major  technical  directorates  to  be  discussed  in  this 

ii'/The  substance  of  this  discussion  is  based  on  information  in  the 
Weapon  System  Acguisition  Guide  and  the  System  Engineering 
Management  Guide. 
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chapter.  This  directorate  is  responsible  for  conductin'g  all  of 
the  analyses  pertaining  to  the  logistics  requirements  of  the  sys 
tem.  The  ILS  Directorate  spans  both  the  activities  associated 
with  designing  the  system  by  AFSC,  and  the  ultimate  responsibil¬ 
ity  of  AFLC  for  supporting  the  system  once  fielded. 

The  ILS  Directorate  is  responsible  for  developing  and 
performing  the  activities  in  the  ILS  Master  Plan  (ILSMP  or  ILSP) 
conducting  the  Logistic  Support  Analysis  ( LSA )  and  in  developing 
and  maintaining  the  LSA  record  ( LSAR ) .  Much  of  this  is  accomp¬ 
lished  in  conjunction  with  the  contractor.  This  covers  a  broad 
territory  and  includes  analyses  related  to  all  of  the  major  ILS 
elements,  as  well  as  areas  of  specialty  engineering  such  as 
reliability  and  maintainability. 

In  addition  to  implementing  the  ILSP  and  integrating 
ILS  considerations  in  systems  engineering  and  development,  this 
directorate  also  develops  and  monitors  budget  inputs;  identifies 
AFLC  support  requirements;  establishes  interfaces  with  the  T&E 
activities  to  integrate  logistics-related  test  results;  and 
implements  the  ILS  management  information  system. 

The  major  ILS  areas  of  concern  to  this  directorate  are: 

®  reliability  and  maintainability  interface; 

•  maintenance  planning; 

•  support  equipment; 

•  supply  support; 

•  packaging,  handling  and  transportation; 

•  technical  data; 

•  facilities; 
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©  manpower  requirements  and  personnel; 

•  training,  training  support  and  devices; 

•  logistics  support  resource  funds; 

•  computer  resources  support; 

•  energy  management; 

•  survivability;  and 

•  ILS  test  and  evaluation. 

An  illustration  of  the  organization  of  a  typical  PO  ILS 
Directorate  is  shown  in  Appendix  C.  Specific  activities  related 
to  logistics  are  discussed  in  Chapter  4.  Additional  information 
on  logistics  is  also  contained  in  Appendix  D. 


CHAPTER  4.  FACTORS  INFLUENCING  MATERIEL  READINESS 

Requirements  Formulation  and  Baseline  Development 
Hardware  and  Software  Design  and  Development 
Test  and  Evaluation 
Integrated  Logistics  Support 
Program  Structuring  and  Manager, .ent 
Suggested  Readings 
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CHAPTER  4.  FACTORS  INFLUENCING  MATERIEL  READINESS 


In  Chapter  3  the  basic  activities  occurring  in  each  phase  of 
most  system  acquisitions  were  reviewed/  as  was  tne  organization 
of  a  typical  Program  Office  (PO).  While  individual  programs  may 
depart  from  the  order  or  arrangement  of  these  activities,  or  the 
organization  of  the  PO,  these  structures  generally  apply  to  the 
vast  majority  of  systems.  At  the  very  least  they  provide  the 
baseline  for  system  acquisition  planning  and  organization. 

The  consideration  of  those  factors,  or  elements,  in  a  pro¬ 
gram  which  relate  to  a  system's  materiel  readiness  requires  one 
to  look  beyond  specific  activities  and  organizations  to  basic 
functions.  It  is  necessary  to  consider  materiel  readiness  as  an 
integrated  capability  to  produce  an  ongoing,  reconstitutable 

ability  to  perform  the  required  mission  when  needed.  This  means 
expanding  on  concepts  such  as  calculated  operational  availability 
to  focus  on  the  comprehensive  maintenance  and  support  capability 
of  the  system.  In  the  acquisition  process  this  capability  is 
represented  largely  by  the  system's  Reliability  and  Maintaina¬ 
bility  (R&M)  and  the  Integrated  Logistic  Support  ( ILS )  system. 

The  system's  reliability  and  maintainability  relate  to  the 
dependability  of  the  system  and  the  ease  of  being  able  to  main¬ 
tain  the  system.  While  they  are  frequently  referred  to  jointly, 
they  represent  different  design  characteristics.  System  design¬ 
ers  must  address  these  different  characteristics  individually, 
recognizing  that  they  are  separate  capabilities. 
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For  example,  while  system  reliability  growth  is  based  on  the 
maturing  of  the  system's  technology,  and  can  be  projected,  no 
techniques  are  currently  available  for  meaningfully  projecting 

and  quantifying  improvements  in  a  system’s  maintainability  beyond 
two  years  past  IOC. 

Both  reliability  and  maintainabiity  are,  however,  perceived 
as  largely  related  to  the  system’s  engineering  and  operational 
characteristics.  By  engineering  characteristics  we  mean  the 
design  of  the  system.  By  operational  characteristics  we  refer  to 
the  system’s  mission  profiles,  life  profile,  and  operational 
environment.  For  the  purposes  of  this  discussion  the  R&M-related 
characteristics  refer  to  activities  in  the  following  areas: 

•  Requirements  Formulation, 

•  Hardware  and  Software  Design  and  Development,  and 

•  Test  and  Evaluation. 

Clearly  these  are  somewhat  artificially  compartmentalized 
since  experts  in  the  field  of  R&M  could  argue  that  many  more  fac¬ 
tors  impact  the  system’s  R&M.  Also,  the  interactions  between 
these  categories  are  so  dynamic  as  to  make  such  segmentation 
arguable.  The  purpose  of  this  chapter  is,  however,  to  very 
generally  discuss  considerations  of  those  activities  that  relate 
to  materiel  readiness.  These  discussions  cannot,  and  are  not 
intended  to  be,  comprehensive  analyses  of  materiel  readiness 
drivers.  Sources  of  analysis  and  additional  information  on  the 

more  specific  aspects  of  this  area  are  referenced  at  the  end  of 
this  chapter. 
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ILS  is  the  collective  term  for  all  of  the  activities 
involved  in  assurring  that  a  system,  once  in  the  field,  can  be 
maintained  in  an  operable  state.  In  the  acquisition  process  it 
represents  the  collection  of  areas  or  functions  which  relate  to 
this  capability.  Logistic  Support  Analysis  (LSA)  is  the  process 
by  which  these  functions  and  areas  are  considered  in  their  own 
right  and  also  interactively  with  other  system  design  require¬ 
ments.  Exhibit  4-1  illustrates  the  nature  of  the  various 
requirements  which  drive  the  system  design.  The  system  require¬ 
ments  are  represented  by  specified  capabilities  regarding: 

•  Producibi 1 ity , 

•  Safety, 

•  Operability, 

•  Maintainability, 

•  Reliability, 

•  Packaging, 

•  Performance,  and 

•  Survivability. 

These  system  requirements  drive  the  ultimate  system  charac¬ 
teristics  that  directly  pertain  to  system  capability.  For 
example,  a  system's  operability  relates  to,  and  is  influenced  by, 
the  quantity  and  quality  of  the  personnel  required  to  operate  and 
maintain  the  system.  The  operability  is  also  related  to  the 
adequacy,  comprehensiveness  and  appropriateness  of  the  training 
those  personnel  receive;  and  the  ease  of  use,  detail,  positioning 
and  placement  of  the  instrument  displays  and  controls  of  the  system. 
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Source:  Navy  Program  Manager's  Guide,  NAVMAT  P-9494,  Naval 

Materiel  Command,  July  1983 


Exhibit  4-1.  SYSTEM  REQUIREMENTS  AND  CHARACTERISTICS 

RELATIONSHIPS 
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All  of  these  system  requirements,  and  the  designing  of  a 
system  to  meet  these  requirements,  is  the  responsibility  of  the 
PO  and  the  associated  contractors.  However,  not  all  of  these 
requirements  relate  to  a  system's  materiel  readiness. 

As  noted  in  Chapter  2,  FACTORS  TO  CONSIDER,  materiel  readi¬ 
ness  can  be  defined  in  a  very  broad,  generic  way,  or  a  very  nar¬ 
row,  specific  manner  (i.e.,  operational  availability.)  For  the 
purpose  of  this  handbook,  readiness  is  considered  as  a  function 
of  system  reliability,  maintainability  and  logistic  supporta- 
bility.  It  is  also  a  result  of  the  successful  and  effective 
interaction  of  requirements  analysis,  system  engineering,  test 
and  evaluation,  configuration  management,  computer  software 
design,  logistic  support  analysis,  and  program  planning,  struc¬ 
turing  and  control. 

Exhibit  4-2  shows  an  overview  of  the  major  acquisition  tech¬ 
nical  activities  and  functional  areas.  The  activities  in  the 
highlighted  area  are  the  technical  functional  areas  having  the 
greatest  impact  on  materiel  readiness: 

•  Requirements  Formulation, 

•  Baseline  Development, 

•  Systems  Engineering  Hardware, 

•  Computer  Software  Peculiar  System  Engineering  Activi¬ 
ties, 

•  Integrated  Logistic  Support,  and 

•  Test  and  Evaluation  Management. 

While  these  areas  do  not  capture  all  of  the  potential  PO 
activities  or  system  acquisition  factors  that  can  be  related  to 
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ACQUISITION  LIFE  CYCLE  TECHNICAL  ACTIVITIES 


Source:  Defense  Systems  Management  College 


Exhibit  4-2.  OVERVIEW  OF  MAJOR  ACQUISITION  TECHNICAL  ACTIVITIES 

AND  FUNCTIONS 
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the  system's  ultimate  materiel  readiness,  they  represent  the 
major  areas  of  technical  concern.  Some  areas  of  activity  are 
more  integrally  involved  than  others  in  the  technical  development 
and  ultimate  nature  of  the  system.  Others  provide  administra¬ 
tive,  managerial  or  procurement  support,  (e.g.,  source  selection, 
contracting/contract  management,  budgeting,  etc.).  While  source 
selection  and  contracting/contract  management  will  surely  impact 
on  the  adequacy  of  the  system  produced  in  the  end,  the  terms  and 
characteristics  of  the  contract  depend  largely  on  the  individual 
program  nature  and  the  circumstances  surrounding  the  acquisition. 

Similarly,  the  budget  process  is  integral  to  retaining  a 
healthy  program,  one  with  sufficient  funds  to  allow  for  the 
development  of  a  well  designed,  logistically  supportable  system. 
However,  the  ultimate  budget  of  a  program  is  determined  largely 
by  forces  outside  of  the  PO  (i.e.,  SAF,  OSD,  Congress).  Although 
the  development  of  budget  strategies  is  outside  the  scope  of  this 
handbook,  the  structuring  of  a  budget  profile  in  terms  of  early 
readiness-related  analysis  is  discussed  later  in  this  chapter. 

This  chapter's  discussion  of  materiel  readiness-related  fac¬ 
tors  focuses  on  the  activities  and  functional  areas  highlighted 
in  Exhibit  4—2.  ( These  areas ,  as  well  as  others  that  relate  to 

concurrency  impacts  on  materiel  readiness/  are  discussed  in 
Chapter  5.  )  These  areas  are  discussed  in  terms  of  particular 
characteristics  which  make  coordinated  planning  for  a  balanced 
approach  to  system  readiness  optimization  particularly  difficult, 
•‘■he  emphasis  in  this  chapter  is  on  considering  those  "factors" 
(i.e.,  activities,  events  or  elements)  in  an  acquisition 
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which  the  Program  Manager  must  be  particularly  aware  of  in 
balancing  materiel  readiness  risks  and  the  other  program  priorities. 

The  technical  functional  areas  highlighted  in  Exhibit  4-2 
have  been  grouped  into  categories  for  the  purposes  of  this  dis- 
cussion.  These  categories  are: 

•  REQUIREMENTS  FORMULATION  AND  BASELINE  DEVELOPMENT, 

which  focuses  on  the  basic  structure  of  the  specifica¬ 
tion,  and  baseline  development  process,  the  reviews  of 
these^  documents  and  the  overall  role  of  these  documents 
in  driving  other  materiel  readiness-related  activities. 

•  HARDWARE  AND  SOFTWARE  DESIGN  AND  DEVELOPMENT,  which 
considers  the  nature?  of  these  activities,  emphasizing 
the  similarities  between  these  two  fields  in  terms  of 
the  need  to  develop  early  mission  and  life  profiles, 
emphasis  on  R&M  characteristics  in  design,  clear  state¬ 
ments  of  requirements,  and  resource  planning  for  fault 
identification  and  corrective  actions. 

®  TEST  AND  EVALUATION,  which  addresses  the  pivotal  role 
played  by  T&E  activities  in  bridging  the  requirements 
and  design  processes  and  is  a  source  of  information  for 
ILS  planning. 

®  INTEGRATED  LOGISTICS  SUPPORT,  which  focuses  on  ILS  plan¬ 
ning,  emphasizing  particular  elements  such  as  develop¬ 
ment  of  the  maintenance  philosophy,  support  equipment, 
manpower,  personnel  and  training  (MPT)  analysis,  and 
the  diagnostic  capability  of  the  system. 

In  addition  to  these  technical  functions,  the  program  struc¬ 
ture  itself  is  discussed  in: 

•  PROGRAM  STRUCTURING  AND  MANAGEMENT,  focusing  on  the 
impact  of  differences  in  the  system  type,  the  system 
operational  environment  and  usage,  technology,  and  the 
current  acquisition  environment,  in  terms  of  system 
readiness . 


REQUIREMENTS  FORMULATION  AND  BASELINE  DEVELOPMENT 


The  cornerstone  of 
description  of  what  the 


system  design  and  acquisition 
new  system  is  required  to  do. 


is  the 
The 
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requirement  for  a  new  system  initially  results  from  the  on-going 

V'-  $*■  *  '• 

Mission  Area  Analysis  conducted  by  OSD,  OJCS,  and  the  Services. 

The  need  to  respond  to  a  new  threat ,  to  expand  capability  in  a 

"i  -*i  **  *  *  < 

given  area,  or  to  capitalize  on  advancing  technology  can  result 

in  the  development  of  a  new  system.  However,  it  is  a  long  way 

ij 

from  developing  a  Justification  for  a  Major  System  New  Start 
(JMSNS)  to  fielding  the  system,  and  along  the  way  the  require¬ 
ments  definition  of  the  new  system  must  be  refined  and  described 
in  increasing  detail. 

Hardware  designers,  software  designers,  logisticians,  sup¬ 
port  equipment  engineers,  reliability  and  maintainability  engi-  H? 
neers,  Program  Managers  and  program  analysts  all  voice  the  same 
thought:  The  requirements  must  be  continuously  reevaluated  for 
reasonableness,  necessity,  and  definition.  There  are  numerous 
reasons  for  emphasizing  the  seriousness  of  requirements  formula¬ 
tion,  particularly  with  respect  to  materiel  readiness. 

First,  the  readiness  of  a  system  is  largely  dependent  on 
what  the  system  is  intended  to  do.  Readiness  relates  to  the 
ability  to  perform  the  mission  the  system  is  intended  to  accom¬ 
plish,  when,  where  and  how  it  needs  to  be  accomplished,  for  the 
duration  of  the  requirement,  under  the  expected  operational  con¬ 
ditions.  This  means  that: 

•  the  mission  or  Missions  the  system  must  perform  must  be 
adequately  thought  through,  effectively  designed,  and 
be  realistic  in  terms  of  performance  requirements? 

•  the  frequency  with  which  the  system  is  expected  to  per¬ 
form  the  mission  (be  operationally  available)  must  be 
realistic  ? 
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•  the  various  environments  in  which  the  system  is 
intended  to  operate  must  be  clearly  defined,  and 
expressed  in  terms  meaningful  to  designers; 

•  the  performance  parameters  must  be  realistically 
defined  and  considered  in  light  of  current  and  advanc¬ 
ing  technology; 

•  the  system  reliability  and  maintainability  goals  and 
requirements  must  be  determined  and  clearly  and  compre¬ 
hensively  described;  and 

•  the  operational  condition  for  the  deployed  system  must 
be  critically  analyzed  not  only  in  terms  of  the  opera- 
tional  missions  but  the  total  system  life  profile. 

In  addition  to  being  rigorously  thought  through,  clearly 
defined,  and  realistically  planned,  system  requirements  must  also 
be  defined  early  in  the  acquisition  process.  The  major  reason 
for  this  is  the  impact  early  design  decisions  have  on  the  post¬ 
deployment  system  status.  Exhibit  4-3  illustrates  the  life  cycle 
cost  expenditure  profile  of  a  typical  weapon  system.  As  can  be 
seen  in  this  exhibit,  over  70%  of  a  system's  life  cycle  costs  are 
fixed  by  decisions  made  in  the  Concept  Exploration  Phase  (i.e., 
by  the  Requirement  Validation  point).  Decisions  regarding  the 
selection  of  the  concept  alternative,  the  maintenance  philosophy, 
and  performance  thresholds  critically  impact  the  ultimate  cost  of 
the  system.  While  approximately  one-half  to  two-thirds  of  the 
life  cycle  costs  are  incurred  after  production  and  deployment,  by 
the  time  the  production  decision  is  made,  approximately  95%  of 
the  costs  have  been  defined.  This  means  the  later  the  require¬ 
ments  are  defined,  the  costlier  it  is.  This  point  will  be  ela¬ 
borated  on  later  in  this  chapter. 

The  second  reason  for  emphasizing  requirements  formulation 
as  a  factor  influencing  materiel  readiness  has  to  do  with  the 
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PERCENT  OF  LlPE-CVCLE  COST 


Source:  Brabson,  G.  Dana  and  Solomond,  John  P. ,  "Readiness  - 

Coequal,"  Concepts  -  Special  Issue:  The  DoD  Acquisition 
Improvement  Program,  Volume  5,  Number  3,  Defense  Systems 
Management  College,  Summer  1982. 


Exhibit  4-3.  LIFE  CYCLE  COST  EXPENDITURE  PROFILE  OF  A 

TYPICAL  WEAPON  SYSTEM 
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nature  of  the  acquisition  phasing.  A  program  cannot  stay  at  the 
same  level  of  development  forever  (although  it  may  sometimes  seem 
that  way).  The  investment  of  government  resources  must  produce 
progress  toward  the  solution  of  the  problem,  or  the  program  will 
be  cancelled.  This  means  the  system  design  must  progress  in 
refinement,  even  it  the  final  requirements  have  not  been  delineated. 

While  the  system  acquisition  process  is  structured  to  reduce 
the  possiblity  of  having  just  such  a  situation  as  lack  of  defini¬ 
tion  of  requirements  (via  the  milestone  reviews),  much  effort  is 
still  expended  in  the  design  process  before  the  requirements  are 
resolved.  Exhibit  4-4  shows  the  progressive  definition  of  system 
specifications  and  the  general  relationship  of  specifications  to 
the  system  baseline.  A  specification  is  "a  document,  intended 
primarily  for  use  in  procurement,  that  accurately  describes  the 
essential  technical  requirements  for  items,  materials,  or  ser¬ 
vices,  including  the  procedures  for  determining  that  the  require- 

1  / 

ments  have  been  met."  Specifications  are  the  single  documented 
statements  of  what  the  system  is  to  do.  In  order  to  be  effective 
they  must  be  refined  from  the  initial  statements  of  operational 
need  (the  user  command-generated  Statement  of  Operational  Need 
(SON )/Required  Operational  Capability  (ROC)/JMSNS)  developed- from 
Mission  Area  Analysis. 

The  first  major  specification  is  the  Type  A,  System  Specifi¬ 
cation,  the  major  requirements  document  produced  in  the  Concept 
Exploration  Phase.  This  specification  includes  a  description  of 
the  technical  and  mission  requirements  of  the  system,  allocates 

V  DoDD  4120.3,  Defense  Standardization  and  Specification  Program. 
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Source:  Navy  Program  Manager  1 s  Guide ,  NAVMAT  P-9494,  Naval 

Materiel  Command,  July  1983 


Exhibit  4-4. 


PROGRESSIVE  DEFINITION  OF 


SYSTEM  SPECIFICATIONS 
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requirements  to  functional  areas  or  configuration  items,  and 
defines  the  interfaces  among  the  functional  areas.  All  of  these 
are  critically  important  to  initiating  Demonstration  and  Valida¬ 
tion  ( D&V )  Phase  activities.  The  system  specification  is  used  to 
establish  the  functional  baseline  configuration  of  the  system. 
Similar  relationships  between  specifications  and  baselines  occur 
in  the  D&V  Phase,  Full  Scale  Development  ( FSD )  Phase  and  the 
Production/Deployment  Phase.  The  specification  is  developed  in 
the  preceeding  phase,  reviewed  and  accepted.  This  specification 
then  forms  the  starting  point  for  the  next  phase  activities, 
which  are  directed  toward  refining  and  developing  a  more  detailed 
specification. 

The  Type  B,  Development  specifications  are  produced  as  a 
result  of  D&V  Phase  activities.  These  specifications  are  in  five 
sections : 


Type  Bl :  Prime  Item  Development, 

Type  B2:  Critical  Item  Development, 

Type  B3:  Non-Complex  Item  Development, 

Type  B4 :  Facility  Development,  and 
Type  B5 :  Computer  Program  Development. 

In  these  specifications  the  system  performance  and  compatibility 
requirements  are  updated,  the  requirements  are  allocated  to  each 
functional  area  and  configuration  item  (Cl),  and  the  objectives 
of  the  functional  baseline  are  translated  into  subsystem  and  Cl 
performance  requirements.  These  specifications  are  the  basis  for 
the  allocated  baseline. 
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The  D&V  Phase  results,  the  Type  B,  Developmental  Specifica¬ 
tions  and  the  allocated  baseline  form  the  basis  for  the  FSD  Phase 
activities.  In  this  phase  the  Type  C,  Production  Specifications 
are  developed,  as  is  the  associated  product  baseline.  (Type  D, 
Process  Specifications,  and  Type  E,  Materiel  Specifications  are 
also  developed  in  this  phase  but  are  not  discussed  here  because, 
for  the  purposes  of  this  handbook,  they  are  less  germane  to  the 
system's  ultimate  materiel  readiness.)  Type  C  Specifications 
also  have  five  sections,  corresponding  to  the  Type  B  sections. 
These  are: 


Type  Cla:  Prime  Item  Product  Function, 

Type  Clb:  Prime  Item  Product  Fabrication  (Part  II), 

Type  C2a:  Critical  Item  Product  Function, 

Type  C2b:  Critical  Item  Product  Fabrication  (Part  II), 
Type  C3:  Non-Complex  Item  Product  Fabrication  (Part  II), 

Type  C4 :  .  Inventory  Item,  and 

Type  C5:  Computer  Program  Product  (Part  II). 

The  Production  Specifications  are  developed  in  the  FSD  Phase  and 
used  in  the  Production  Phase  as  the  basis  for  actually  manufac¬ 
turing  the  system.  The  Production  Specification  provides  all  of 
the  detail  below  the  system  level  (i.e.,  subsystem,  configuration 
item,  data  for  each  level  of  detail  in  the  WBS )  necessary  to  per¬ 
mit  economical  procurement  of  the  functional  elements  that,  when 


assembled  will  produce  a  system  able  to  function  according  to  the 
stated  requirements.  The  sections  of  the  Production  Specifica¬ 
tion  are  divided  into  two  parts:  the  Functional  or  Performance 
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Specification  (Type  Cla)  and  the  Fabrication  or  Design  Specifica¬ 
tions  (Types  Clb,  C2a,  C3,  C4  and  C5). 

Whij.e  the  specifications  and  baselines  are  clearly  closely 
associated,  they  fulfill  different  roles  in  the  program.  The 
specifications  translate  the  Air  Force's  system  requirements  into 
the  design,  are  developed  by  contractors,  reviewed  by  the  Air 
Force  and  managed  primarily  through  the  systems  engineering  func¬ 
tion.  Baselines  are  the  mechanism  the  Air  Force  uses  to  track 
and  control  the  status  of  the  configuration,  through  the  Config¬ 
uration  Management  function. 

As  can  be  seen  from  this  brief  description  of  the  require¬ 
ment/specification/baseline  development  stream,  the  definition 
of  the  system's  requirements,  and  the  review  of  the  specifica¬ 
tions  produced  to  respond  to  these  requirements  must  be  rigor¬ 
ously  monitored.  Exhibit  4-5  illustrates  the  major  reviews  of 
the  specifications  in  each  phase.  The  major  types  of  technical 
reviews  or  audits  in  the  systems  acquisition  process  are  the 


following : 


.±/ 


The  Systems  Requirements  Review  ( SRR ) ,  in  the  Concept 
Exploration  Phase,  reviews  the  interpretation  of  the 
system  requirements  in  the  functional  baseline.  The 
review  may  be  performed  several  times  during  the  CE 
Phase  and  in  the  beginning  of  the  D&V  Phase. 

The  System  Design  Review  (SDR)  reviews  the  system 
documentation  developed  in  the  D&V  Phase,  assesses  the 
degree  of  acompl ishment  of  the  System  Engineering 
activities  and  determines  if  it  is  sufficient  to  pro¬ 
ceed  into  the  preliminary  design  of  selected  solutions 
f  >r  the  allocated  baseline.  It  is  performed  as  a  final 
review  of  the  D&V  Phase  products  or  as  the  initial 
review  in  the  FSD  Phase  (for  systems  not  requiring  a 
formal  validation  phase). 


Ipcrl-ftnn^1^1^  .have  been  based  on  descriptions  contained  in 
AtSLP  a u U  —  3 ,  A  Guide  to  Program  Management. 
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System  Engineering  Management  Guide, 

30  October  1983 


Defense  Systems  Management 


College , 


Exhibit  4-5.  DESIGN  REVIEW  AND  BASELINE  DEVELOPMENT  RELATIONSHIP 


The  Preliminary  Design  Reviews  ( PDR )  are  multiple 
reviews  conducted  throughout  the  FSD  Phase  of  the 
status  of  each  configuration  item.  PDRs  evaluate  the 
progress,  consistency  and  technical  adequacy  of  the 
selected  design  and  test  approach  and  established  com- 
patibility  with  the  preliminary  design.  A  successful 
PDR  is  necessary  for  each  Cl  before  proceeding  into 
detailed  design.  However,  it  does  not  have  to  be  based 
on  an  approved  allocated  baseline. 

The  Critical  Design  Reviews  (CDR)  are  conducted  for 
each  Cl  and  computer  program  configuration  item  (CPCI) 
to  determine  the  acceptability  of  detail  design,  per¬ 
formance,  and  test  characteristics  depicted  in  the 

product  specification,  accompanying  drawings,  and 
other  engineering  documentation.  CDRs  are  conducted  in 
the  FSD  Phase.  For  a  CPCI,  these  reviews  are  performed 
after  the  coding  is  completed  and  are  a  formal  review 
of  the  software  design. 

The  Functional  Configration  Audit  ( FCA )  differs  from 
the  technical  reviews  in  that  it  measures  the  degree  of 
compliance  of  the  actual  Cl  against  the  Part  I  develop¬ 
mental  specification.  It  involves  reviewing  test  data 
to  verify  performance  for  each  Cl.  An  FCA  of  the  CPCIs 
is  necessary  before  the  CPCI  product  baseline  can  be 
established.  Included  in  the  FCA  is  an  analysis  by  the 
contractor  of  the  requirements  that  could  not  be  met, 
proposed  solutions,  and  a  description  of  the  Engineer¬ 
ing  Change  Proposals  (ECP)  which  have  been  incorporated 
and  tested.  The  test  data  are  compared  to  the  test 
plans  and  procedures  to  ensure  completeness  and  accur¬ 
acy.  The  FCA  occurs  at  the  end  of  the  FSD  Phase. 

The  Physical  Configuration  Audit  (PCA)  is  the  formal 
examination  of  the  "as  built"  version  of  the  CIs  and 
CPCIs  against  the  technical  documentation  to  establish 
the  product  baseline.  It  includes  a  detailed  audit  of 
engineering  drawings,  specifications,  technical  data 
and  tests  used  in  the  production  of  hardware  CIs  and  a 
detailed  audit  of  technical  descriptions,  flow  charts, 
listings,  manuals  and  handbooks  for  CPCIs.  The  PCA 
usually  occurs  in  the  Production  Phase. 

The  Formal  Qualification  Review  ( FQR )  is  the  final 
major  review  of  the  hardware  CIs  and  the  software  CPCIs 
to  verify  that  the  actual  performance  of  the  Cl  com¬ 
plies  with  the  Part  I  developmental  specification  and 
to  identify  the  test  reports  and  data  that  document  the 
CPCI  qualification  tests.  The  FQR  can  occur  in  conjunc¬ 
tion  with  the  FCA,  or  if  necessary,  be  delayed  until 
after  PCA.  The  FQR  is  the  point  when  the  Cl  is 
officially  entered  into  the  Government  inventory. 


4-18 


The  scheduling  of  each  of  these  audits  and  reviews,  and  the 
exact  structure  and  detail  of  the  reviews,  is  the  responsibility 
of  the  PM.  It  is  his  or  her  responsibility  to  assure  that  the 
program  activities  are  producing  a  specification  and  baseline 
that  responds  to  and  delineates  the  requirements.  Clearly,  it  is 
possible,  but  very  dangerous,  to  wait  for  the  Preliminary  Design 
Review  to  strenuously  evaluate  the  system  requirements.  By  that 
time,  design  efforts  and  program  resources  will  have  been  expended 
in  two  phases,  possibly  inefficiently.  The  lesson  shown  in 
Exhibit  4-3  of  the  life  cycle  cost  commitment  and  expenditure 
profile  points  out  the  potential  danger  of  committing  system 
resources  in  a  period  when  the  requirements  may  not  be  well 
defined. 

System  requirements  drive  all  of  the  other  areas  which 
relate  to  the  materiel  readiness: 

•  Hardware  Design, 

•  Software  Design, 

•  Test  and  Evaluation,  and 

•  Integrated  Logistic  Support. 

The  emphasis  on  front-end  analysis  of  supportabi lity  issues,  as 
evidenced  by  the  Acquisition  Improvement  Program  (AIP)  reinforces 
the  need  for: 

•  early  definition  of  requirements? 

•  continuous  review  of  requirements  throughout  the  acqui¬ 
sition; 

•  detailed  descriptions  of  requirements  developed  by  the 

Air  Force  PO  functional  areas  (Systems  Engineering, 
Configuration  Management,  Software  Design,  Test  and 
Evaluation,  ILS,  etc. )  in  concert  with  the  contractors; 


•  effective  measures  for: 

monitoring  progress, 
identifying  areas  of  uncertainty, 

clearly  defining  disconnections  in  relating  system 
specifications  development  to  system  requirements, 

communicating  updated  information  to  all  of  the 
functional  areas;  and 

•  integration  of  all  of  the  functional  system  require¬ 
ments  for  the  total  life  cycle  of  the  system. 

It  is  fundamentally  dangerous  to  assume  that  problems,  or 
holes,  in  the  requirements  definition  can  be  fixed  later  in  the 
program,  without  a  significant  expenditure  of  resources,  poten¬ 
tial  delay  in  program  completion  and  an  associated  failure  to 
provide  an  adequate  support  system.  The  later  discussion  of 
software  development  problems  painfully  applies  to  most  of  the 
other  areas  in  the  system  acquisition  process. 

The  requirements  formulation  process  has  been  discussed  here 
as  the  driving  entity  behind  the  detailed  activities  in  the  tech¬ 
nical  functions  that  we  feel  most  heavily  impact  materiel  readi¬ 
ness.  Each  of  these  areas  relates  to  one  another  as  well.  For 
example,  the  hardware  and  software  designs  must  be  compatible 
with  each  other,  must  be  adequate  for  not  only  the  primary  mis- 

V 

sion  system  (aircraft,  missile,  communications,  etc.),  but  also 
the  system  support  equipment.  They  must  be  designed  to  optimize 
their  supportabi lity  characteristics  and  the  design  characteris¬ 
tics  and  requirements  must  be  integrated  with  the  ILS  and  Test 
and  Evaluation  efforts. 

Test  and  Evaluation  plans,  procedures  and  results  must 
reflect  the  system  requirements,  be  planned  and  applied  in  each 


4-20 


* 


* 


stage  of  development,  and  must  be  designed  to  respond  to,  and 
test  for,  the  required  hardware  and  software  capabilities.  The 
results  of  the  T&E  activities  must  feed  into  ILS  planning  and 
analysis  since  they  impact  on  the  ultimate  supportability  of  the 
system. 

The  ILS  activities  are  critical  to  developing  a  "materially 
ready"  system.  These  activities  cannot  be  the  "late  sister"  in 
the  design  process,  deferred  until  the  major  hardware  and  soft¬ 
ware  decisions  are  made.  Deferral  of  ILS  considerations,  partic¬ 
ularly  of  elements  such  as  manpower  and  personnel,  training  plan¬ 
ning  and  devices,  determination  of  the  maintenance  philosophy  and 
design  and  development  of  the  support  equipment,  can  result  in  a 
system  without  sufficient  qualified  personnel,  which  cannot  be 
adequately  maintained. 

In  the  discussions  of  these  areas  and  program  planning  that 
follow,  the  intent  is  to  address  activities,  and  problems  asso¬ 
ciated  with  completing  these  activities,  that  directly  relate  to 
the  issue  of  materiel  readiness.  It  is  impossible  for  these 
discussions  to  cover  all  possible  variations  on  activities  and 
the  many  ramifications  and  difficulties  that  can  occur.  That  has 
not  been  attempted.  Rather,  these  discussions  are  intended  to 
raise  issues  that  PMs  and  their  staffs  should  use  as  starting 
points  in  their  consideration  of  how  decisions  they  make  on  their 
program  can  influence  materiel  readiness.  At  the  end  of  this 
chapter  is  a  list  of  other  sources  that  examine  these  issues  and 
would  be  useful  to  consult. 
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HARDWARE  AND  SOFTWARE  DESIGN  AND  DEVELOPMENT 


Today's  new  systems  are  highly  automated,  technologically 
advanced,  and  rely  a  great  deal  on  the  symbiotic  relationship  of 
hardware  and  software,  such  as  found  in  advanced  avionics  systems, 
built  in  test  equipment  (BITE),  automatic  test  equipment 
(ATE),  and  other  mission  and  support  systems.  This  concentration 
on  the  use  of  embedded  computer  resources  ( ECR )  in  new  systems 
means,  among  other  things,  that  materiel  readiness  must  be 
thought  of  as  being  a  product  of  not  only  successful  hardware  and 
software  development  efforts,  but  also  the  successful  development 
of  these  systems  in  a  joint  effort. 

Hardware  and  software  development  share  many  things  in  com¬ 
mon,  however,  the  least  common  of  these  is  the  cost  of  developing 
and  testing  them.  Currently,  in  advance  technology  systems,  the 
cost  of  developing  the  system  hardware  is  approximately  equal  to 
the  cost  of  developing  the  software  to  operate  the  system.  How¬ 
ever,  this  is  changing.  Exhibit  4-6  illustrates  three  views  of 
the  trend  in  the  relationship  of  software  and  hardware  costs  in 
the  near  future.  The  rising  financial  and  budgetary  emphasis 
also  has  a  similar  trend  in  the  cost  of  maintaining  the  software, 
as  shown  in  Exhibit  4-7.  All  this  means  that  materiel  readiness 
is,  to  a  large  extent,  going  to  increasingly  relate  to  the  soft¬ 
ware  side  of  the  system.  This  does  not  mean  that  hardware  is  not 
an  issue.  Rather,  the  system  must  be  perceived  as  a  whole.  As 
technology  and  hardware  reliability  and  maintainability  receive 
greater  emphasis  as  readiness  drivers,  software  R&M ,  and  the 
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Source:  Grove,  H.  Mark,  "DoD  Policy  for  Acquisition  of  Embedded 

Computer  Resources,"  Concepts:  Special  Issue  -  Managing 
Software,  Volume  5,  Number  4,  Defense  Systems  Management 
College,  Autumn  1982 

Exhibit  4-6.  TRENDS  IN  SOFTWARE  AND  HARDWARE  COSTS 


Source:  Bunyard,  Maj  Gen  Jerry  M ,  and  Coward,  James  M. 

"Today's  Risks  in  Software  Development  -  Can  They  be 
Significantly  Reduced?",  Concepts:  'Special" Issue- - 
Managing  Software,  Volume  5,  Number  4,  Defense  Systems 
Management  College,  August  1982 


Exhibit  4-7.  HARDWARE- SOFTWARE  DEVELOPMENT  AND  MAINTENANCE 

COST  TRENDS 
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interactions  and  dependencies  between  the  two,  must  be  recognized 
and  managed.  Some  of  the  more  technologically-oriented  relation¬ 
ships  among  system  R&M  drivers,  support  elements,  and  management 
structures  have  be  discussed  in  a  recent  study  by  OSD  and  the 
Institute  for  Defense  Analyses.-^ 

There  are  similar  threads  that  run  throughout  any  discussion 
of  materiel  readiness  and  the  factors  that  influence  it,  regard¬ 
less  of  the  specific  nature  of  the  system.  Many  of  these  have 
been  long  understood  in  relation  to  hardware  design.  The  expan¬ 
sion  of  system  priorities  beyond  mission-specific  priorities  to 
support  concerns,  and  the  advances  in  automation  mean  that  pre¬ 
viously  learned  lessons  will  be  relearned  in  a  new  area.  One  of 
the  main  realities  to  remember  is  that  every  piece  of  the  system 
(hardware  and  software)  that  is  operated  must  be  maintained. 

Increasing  system  readiness  means  focusing  on  three  major  aspects 
of  system  capability: 

•  Reliability  -  the  frequency  with  which  the  system 
fails; 

•  Maintainability  -  the  ease  with  which  the  system  can  be 
repaired;  and 

•  Suppor tabi 1 ity  —  the  extent  of  the  resources  required 
to  keep  the  system  operational. 

Software,  like  hardware,  must  be  considered  not  only  from 
the  perspective  of  the  system  mission  (operations  and  applica¬ 
tions  programs)  but  also  the  extensive  support  software  which 
must  be  developed.  Exhibit  4-8  illustrates  the  iceberg  relation¬ 
ship  of  system  and  support  software.  All  of  this  software  is 

-/  IDA/OSD  Reliability  and  Maintainability  Study,  Institute  for 
Defense  Analyses,  November  1983. 
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Mciivaine,  Paul  J.,  "Software  Logistics:  A  Sleeping 
Giant  ,  Concepts:  Special  Issue  -  Managing  Software, 

Volume  5,  Numbef  4,  Defense  Systems  Management  College, 
Autumn  1982 


Exhibit  4-8.  TYPES  OF  SOFTWARE 


interacting  with  hardware  and  the  total  system  must  be  designed 
for  readiness.  This  need  to  consider  operational  and  support 
requirements  of  both  the  hardware  and  software  means  that  the 
design  and  development  efforts  must  keep  the  total  life  cycle 
cost  of  the  system  in  perspective. 

As  mentioned  earlier,  hardware  and  software  design  and 

,!  1 

development  share  many  similar  characteristics  relating  to 
mater iel  readiness.  The  first  is  the  need  to  have  effective? 
consistent,  rigorously  validated  requirements,  specifications, 
and  baselines;  early,  continuous,  thorough  reviews  of  require¬ 
ments  and  design;  and  an  effective  and  comprehensive  fault  iden¬ 
tification  and  corrective  action  process  in  the  design  and  devel 
opment  process.  A  key  step  in  achieving  these  objectives  is 
establishment  of  Mission  and  Life  Profiles  as  soon  as  possible 
after  program  initiation.  The  influence  of  these  profiles  is 
shown  in  Exhibit  4-9. 

A  mission  profile  lays  out  in  detail  the  sequence  of  activi 
ties  and  set  of  capabilities  the  system  must  be  able  to  perform 
and  withstand  during  a  given  mission.  If  a  system  has  more  than 
one  mission,  a  Mission  Profile  musp  be  developed  for  each 
mission.  These  profiles  are  major  inputs  to  the  requirements 
formulation  process  and  must  be  regularly  reviewed  and  updated 
based  on  changes  in  the  intended  mission  of  the  system.  A 
mission  profile  for  an  air  launched  missile  would  take  into 
consideration  such  factors  as  those  found  in  Exhibit  4-10. 

^  kif®  Profile  expands  on  the  Mission  Profile  by  incorpora¬ 
ting  a  time-phased  description  of  the  events  and  environments 
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IDA/OSD  Reliability  and  Maintainability  S tudy , 

for  Defense  Analyses,  November  1983  - 
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Exhibit  4-9.  MAJOR  PROGRAM  ELEMENT  INTERRELATIONSHIPS 

AND  DEPENDENCIES 


4-28 


1.  Proposed  logistics  cycle 

2.  Means  of  transportation  (truck,  railroad,  dolley,  etc.) 
of  the  weapon  from  one  location  to  the  next 

]■ 

3.  Range  of  time  spent  at  each  location  and  the  environ¬ 
ment  encountered  there 

4.  •  Identity  of  carrying  vehicle  (ship  or  aircraft)  on 

which  the  weapon  will  be  stored  or  carried,  or  from 
which  it  will  be  launched 

5.  Anticipated  locations,  in  or  on  the  carrying  or  launch¬ 
ing  vehicle,  where  the  weapon  will  be  carried  or 
launched  from,  and  the  mix  of  stores  carried  by  that 
vehicle 

6.  Anticipated  combat  tactics  employed  by  the  carrying  or 
launching  vehicle  and  its  maneuvering  characteristics 
and  limitations  (speed,  altitude,  depth,  etc.) 

7.  Anticipated  mission  profile  of  carrying  or  launching 
vehicle 

8.  Anticipated  operational  deployment  areas  of  the  carry¬ 
ing  or  launching  vehicle  (sea,  land,  desert,  arctic, 
worldwide ,  etc. ) 

9.  Required  life  span  of  the  candidate  weapon  components 
(storage  life,  service  life,  number  of  flights,  etc.) 

10.  Operational  experience  pf  existing  similar  weapons 


SOURCE:  Navy  Program  Manager's  Guide,  NAVMAT  P-9494,  Naval 

Material  Command,  July  1983. 


Exhibit  4-10.  EXAMPLE  OF  MISSION  PROFILE  CONTENTS  FOR  AN  AIR 
LAUNCHED  MISSILE 
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that  an  item  will  experience  from  cradle  to  grave.  The  Life 

is  critical  in  requirements  formulation  and  subsequent 
design  activities  since  much  of  the  total  life  of  a  system  is 
spent  in  a  non-operational  mode.  The  following  table,  from  the 
IDA/OSD  Reliability  and  Maintainability  Study,  illustrates  this 
point . 


Duration  of  Non-Mission  Periods  in  A  System's  Life 


Mission 

Life  Time 


%  of  Life  in 
Non-M ission 
Time 


Missile  Electronics  10-15  years  30  min. 
Aircraft  Electronics  15  years  4,000  hrs. 


99.9% 

96.9% 


Non  mission  activities  can  stress  the  system,  significantly 
reducing  readiness.  Sources  of  non-mission  stress  are: 

•  transportation, 

•  handling, 

•  storage,  and 

•  test  or  checkout. 

Exhibit  4-11  is  an  example  of  a  Life  Profile  for  a  missile  sys¬ 
tem.  The  Life  Profile  must  be  developed  to  represent  the 
extremes  in  stress  and  environronet  to  which  the  system  may  be 
subject  in  order  to  design  and  develop  the  system  to  withstand 
extremes  in  temperature,  shock,  vibration  and  humidity.  These 
profiles  must  be  developed  in  conjunction  with  life  cycle  cost 
and  design  to  cost  goals  in  a  realistic  manner  in  order  to  permit 
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Source:  Navy  Program  Manager's  Guide,  NAVMAT  P-9494,  Naval 
Material  Command,  July  1983 

Exhibit  4-11.  SYSTEM  LIFE  PROFILE 
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development  of  realistic  and  achievable  contractual  statements  of 
work,  as  well  as  functional  area  plans. 

Another  major  readiness-related  consideration  in  software  as 
well  as  hardware  design  is  the  emphasis  on  early  efforts  in 
design.  The  increase  in  the  duration  of  the  acquisition  of  soft¬ 
ware  paces  that  of  hardware.  The  two  cannot  be  developed 
independently.  Many  software  designers  increasingly  state  that 
software  design  must  be  given  the  same  early  and  consistent 
attention  as  hardware,  in  a  disciplined  software  development 
effort.  The  following  quote  amplifies  this  theme. 

Emphasis  continued  to  be  given  to  cost  for  development, 
with  little  attention  being  given  to  the  overall  life-cycle  cost. 
This  leads  to  a  further  rise  in  maintenance  costs  due  to  latent 
errors  in  the  software  and  inflexible  software  design,  as  well  as 
insufficient  documentation  and  support  software  for  maintenance. 
Support  packages,  e.g.,  simulations  and  test  tools,  are  often 
treated  as  throwaways  rather  than  major  deliverable  items  that 
^aci^ltate  development  and  maintenance.  More  emphasis  must 

be  placed  on  the  required  discipline  and  rigor  early  in  develop¬ 
ment  to  ensure  quality,  including  maintainability  of  the  software 
throughout  the  system  life  cycle. 

"...What  are  the  primary  problem  areas  today  that  lead  to 
schedule  slippages,  cost  overruns,  or  a  software  product  that 
falls  short  of  its  desired  goals?  Some  of  these  problem  areas 
are  original  requirements  that  are  incomplete  and/or  validated; 
software  design  that  is  not  traceable  to  the  requirements  and 
diverges  during  development;  software  code  that  is  not  maintain¬ 
able  due  to  poor  enforcement  of  standards;  documentation  of  the 
system  that  does  not  reflect  as-built  code;  software  that  is 
insufficiently  tested;  and  timing  and  storage  budgets  that 
are  exceeded."  2./ 

4 

Exhibit  4-12  shows  the  average  duration  (in  months)  of  the  acqui¬ 
sition  of  avionics  systems. 


—  Bunyard,  Maj  Gen  Jerry  M.  and  Coward,  James  M.  ,  "Today's  Risks  in 
Software  Development  -  Can  They  Be  Significantly  Reduced?", 
Concepts;  Special  Issue  -  Managing  Software,  Volume  5,  Number  4, 
Defense  Systems  Management  College,  Autumn  1982. 


4-32 


MONTHS 


4-6 


9-12 


12-18 


24-30 


36-42 


-SOFTWARE 
- PART 


DESIGN- 
1  SPECS- 


PDR 


PRELIMINARY  _ 

“coding  begins" 


■PART  II  SPECS- 


I 

U) 

u> 


CDR  (#1) 
SUPPORT 
TOOLS 
NEEDED 


PRINCIPAL  CODING 

—  &  SOFTWARE  - 

INTEGRATION 


CDR  (N) 

MATURE  COMPILER/SUPPORT 
TOOLS  AND  COMPUTER  HARDWARE 
REQUIRED 


SYS  INTEGRATION 
LAB  (SIL) 


HARDWARE  AND 
SOFTWARE  INTEGRATED 


AVIONICS  SUBSYSTEMS 
HARDWARE  REQUIRED 


FIRST  FLIGHT 


Exhibit  4-12.  AVIONICS  SOFTWARE  DEVELOPMENT  SCHEDULE 


■p 


As  with  hardware  design  and  development,  errors  not  caught 
in  the  early  concept  development  and  design  effort  become  embed¬ 
ded  in  the  software  only  to  have  much  greater  cost,  operational 
and  maintenance  impacts  later  on.  This  upward  trend  in  the  cost 
and  design  impacts  of  error  detection  and  correction  late  in  the 
development  process  is  shown  in  Exhibit  4-13,  which  shows  three 
trends  in  the  impact  of  errdr  detection  in  the  software  life 
cycle. 

Focusing  on  early  detection  of  errors  means  shifting  empha- 

f*T  j 

sis  to  the  specification  development  portion  of  the  software 
development,  just  as  with  the  development  of  hardware.  Exhibit 
4-14  shows  this  process  in  terms  of  activities  relating  to  error 
introduction  and  correction  and  how  system  verification  and  vali¬ 
dation  can  be  brought  to  bear  to  improve  this  process.  The 
following  quote  summarizes  the  managerial  orientation  the  Program 
Manager  should  have  in  managing  software. 

"There  also  can  be  a  nightmare  of  system  performance 
failure,  cost  ovenun,  schedule  slippage,  and  even  loss  of  pro¬ 
gram  control  awaiting  the  program  manager  who  does  not  manage 
software  development  with  the  skill  and  to  the  same  extent  that 
he  manages  hardware  development.  Equal  emphasis  on  software  and 
hardware  should  apply  from  the  beginning.  The  ECR  must  be 
evaluated  as  to  its  supportabi 1 ity :  common  high  order  language, 
availability  of  compilers,  transportability  of  coding,  documenta¬ 
tion  deliverables,  etc.  Universal  test  equipment  availability 
for  the  hardware  and  for  the  software  are  features  which  should 
also  be  evaluated.  In  short,  system  software  should  be  treated 
as  a  vitally  important  configuration  item  just  as  much  as  is  the 
engine  in  an  aircraft  weapon  system  selected  for  development.... 

"Software  management  has  been  recognized  by  successful  pro¬ 
gram  managers  as  requiring  early,  intensive,  continuing  manage¬ 
ment.  There  is  no  program  phase  that  is  too  early  for  concern 
with  software.  In  fact  if  the  software  standards  to  be  utilized 
are  not  identified  as  early  as  issuance  of  the  system  solicita¬ 
tion,  some  form  of  disaster  is  highly  probable.  Hardware  devel¬ 
opment  that  is  allowed  to  proceed  in  advance  of  software 
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Exhibit  4-13a. 
OVERVIEW  OF  POTENTIAL 
LIFE-CYCLE  DEFECTS  IN 
PROGRAMMING 


EQrf|4i(i 


Exhibit  4-13b. 

CATCHING  SOFTWARE  ERRORS 
EARLY:  PROJECT  RESULTS 


Exhibit  4-13c. 

CATCHING  SOFTWARE  ERRORS 
LATE:  THE  COST 
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Source:  Bunyard,  Maj  Gen  Jerry  M.  and  Coward,  James  M., 

"Today's  Risks  in  Software  Development  -  Can  They  Be 
Significantly  Reduced?",  Concepts:  Special  Issue  - 
Managing  Software,  Volume  5,  Number  4,  Defense 
Systems  Management  College,  Autumn  1982 


Exhibit  4-13.  LIFE  CYCLE  IMPACTS  OF  SOFTWARE  ERROR 
DETECTION  AND  CORRECTION 
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Exhibit  4-14b . 


SOFTWARE  VERIFICATION  AND 
VALIDATION 


Source : 
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Exhibit  4-14.  SOFTWARE  DEVELOPMENT,  VERIFICATION  AND  VALIDATION  ACTIVITIES 
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decisions  will  generally  constrain  system  design.  Program 
decisions  in  good  designs  are  made  with  engineering  balance 
between  hardware  and  software  considerations. 

"Program  managers  are  enjoined  in  DOD-STD-1679A  to:  .(1)  make 
extraordinary  efforts  in  the  development  phase  to  ensure  maximum 
reliability  and  maintainability  of  software;  (2)  ensure  software 
is  designed  to  facilitiate  efficient  changes  (even  at  the  expense 
of  technical  design  efficiency,  if  necessary);  and  (3)  design 
software  which  is  strongly  influenced  by  factors  which  will 
reduce  life-cycle  cost,  particularly  those  standards  relating  to 
design,  languages,  intersystem  and  intrasystem  interfaces. 

"These  guidelines  emphasize  that  software  should  be  designed 
to  make  changes  easy — and  that  it  is  worth  extra  time,  effort, 
and  resource  expenditure  during  the  development  phase  to  make  the 
software  reliable,  maintainable,  and  less  costly  over  its  opera¬ 
tional  life  span.  These  guidelines,  if  followed,  can  help  to 
avoid  major  problems  in  system  oper at  ion . . . . I f  software  is 
allowed  to  become  complicated  and  overly  refined,  a  simple  change 
in  one  or  two  parameters  can  demand  that  major  software  segments 
be  replaced.  Careful  design  (including  memory  medium)  with 
regard  for  probable  programming  change  requirements  can  simplify 
the  software  change  problem  throughout  the  system's  service  life. 
Costs  of  updating  and  maintaining  software  during  the  system's 
service  life  can  be  controlled  only  if  the  program  manager  forces 
the  design  in  a  reliable,  maintainable  direction. 

"....There  are  no  golden  rules  which,  if  followed,  will 
avoid  software  problems  in  the  development  of  a  software  inten¬ 
sive  system.  Treating  software  development  as  an  effort  equally 
important  to  and  concurrent  with  hardware  development  has 
produced  the  best  results.  Paying  too  little  attention  to,  or 
neglect  of  software  has  produced  system  failures.  As  in  all 
aspects  of  program  management,  in  software  development  as  in 
total  system  development,  there  is  no  substitute  for  understand¬ 
ing  what  must  be  done  and  doing  it  at  tha  proper  time  in 
the  acquisition  process. "5/ 

A  critical  factor  influencing  the  PM ' s  ability  to  develop 
effective,  error-free,  easily  maintained  software  is  common 
throughout  system  design,  operation,  and  maintenance:  the  dwind¬ 
ling  supply  of  adequately  trained  and  experienced  software 
programmers  and  designers.  The  explosion  of  computer  applica¬ 
tions  in  industry  competes  for  the  pool  of  experienced  software 


£/  Navy  Program  Manager's  Guide,  NAVMAT  P-9494,  Naval  Material 
Command,  July  1983. 
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designers.  This  combines  with  a  specific  characteristic  of 
software  design  -  it  is  highly  sequential  in  nature,  and  allows 
for  only  limited  application  of  concurrency  or  other  time¬ 
shortening  devices. 

The  sequence  of  activity  shown  in  several  of  the  preceeding 
illustrations  limits  the  overlapping  of  activities.  Two  ap¬ 
proaches  for  attempting  to  achieve  shorter  acquisition  times  and 
increased  design  effectiveness  are  adding  more  people  to  tasks 
and  maximizing  standardization  in  design  elements.  The  former  is 
sometimes  referred  to  as  "the  Mongolian  Hordes  versus  Super 
Programmer  Approach".-/  This  approach  has  limited  effectiveness 
due  to  a  number  of  reasons: 

•  the  difficulty  in  acquiring  and  retaining  skilled 
programmers/designers ; 

•  the  manloading  limits  on  tasks,  largely  driven  by  how 
much  the  task  can  be  compartmentalized;  and 

•  the  sequential  nature  of  the  process  does  not  allow  for 
significant  concurrent  progression. 

While  concurrency-related  activities  are  discussed  in  more 
detail  in  Chapter  5,  this  aspect  of  concurrency  and  software 
design  has  been  pointed  out  to  illustrate  that  while  there  are 
similarities  among  hardware  and  software  design  and  development, 
there  are  also  differences.  Part  of  the  reason  for  these  differ¬ 
ences  has  to  do  with  the  fundamental  nature  of  hardware  and 
software . 


_  This  concept  as  well  as  problems  associated  with  software  sched- 
tit  dlscussed  in  Frederick  P.  Brooks,  Jr.'s,  The  Mythical 
5F  this  ch - ln  Software  Engineering,  referenced  at  the  end 
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It  is  very  difficult  to  measure  progress  in  software  design, 
and  it  is  not  practical  to  use  the  same  techniques  to  measure 
progress  as  used  in  hardware.  Just  as  reliability  and  maintain¬ 
ability  goals  must  be  tailored  to  the  specific  type  of  system,  so 
also  measures  of  progress  must  be  developed  for  software,  as  they 
have  been  for  hardware.  Among  other  things,  this  means  a  shift 
in  perspective  regarding  software  design  from  thinking  of  it  as 
an  art,  with  each  program  an  example  of  individual  craftsmanship  - 
to  a  disciplined  body  of  knowledge  with  standards,  consistent 
architecture,  language,  and  design  and  documentation  practices. 
This  emphasis  on  discipline  and  consistency  will  increase  the 
ultimate  maintainability  and  reliability  of  the  software  and, 
therefore,  the  total  system. 

The  use  of  standards  relates  to  the  second  approach  for 
increasing  the  effectiveness  of  design  approaches:  the  use  of 
modular  designs.  This  approach  is  a  common  interest  in  both 
software  and  hardware.  Much  effort  has  been  spent  in  recent 
years  in  exploring  and  encouraging  the  use  of  common  components, 
design  modules,  standardized  design  elements  and  the  application 
of  standards  to  both  hardware  and  software  design.  The  intention 
is  to  increase  system  readiness  and  decrease  acquisition  time  and 
cost.  The  trade-off  is  between  performance  flexibility  and 
inter-system  commonality,  increased  maintainability,  and  design 
reliability.  From  a  hardware  perspective  there  has  been  a  very 
slow  building  of  enthusiasm  for  this  idea.  In  software  develop- 

v 

ment  there  has  been  a  significant  move  toward  revising  the 
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relevant  military  standards  to  standardize  the  high  order  lan¬ 
guages  (e.g. /  DoD-wide  use  of  ADA),  system  architecture,  and  the 
design  and  development  of  reusable  code  modules.  Continued 
exploration  of  all  of  these  will  increase  the  capability  of 
designers  and  managers  to  design  reliability  and  maintainability 
into  the  system.  These  are  just  a  few  of  the  major  elements  in 
hardware  and  software  design  impacting  materiel  readiness.  .The 
reader  should  continue  to  explore  these  areas,  and  should  review 
the  references  at  the  end  of  the  chapter.  The  following  discus¬ 
sion  focuses  on  test  and  evaluation  activities  related  to 
materiel  readiness. 

TEST  AND  EVALUATION 


A  major  factor  in  a  system's  readiness  when  fielded  is  the 
effectiveness  of  the  Test  and  Evaluation  (T&E )  program  applied 
in  the  acquisition.  T&E  plays  a  pivotal  role  in  that  it: 

•  evaluates  the  design  in  terms  of  how  it  satisfies  the 
system  requirements, 

*  ment°effortserminati0n  °f  ^  successof  the  develop- 

*  capabilityeterminati°n  °f  ^  system's  operational  ' 

•  supports  the  system  qualification  process,  and 

provides  actual  information  on  system  operational  and 
maintenance  capabilities  to  feed  into  ILS  planning. 

A  difficulty  intrinsic  in  the  effective  use  of  testing  and 

evaluation  in  the  system  design  is  the  fact  that  there  must  be 

something  to  test  in  order  to  obtain  results.  Test  results  then 
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become  a  significant  decision  driver,  particularly  in  transition¬ 
ing  from  FSD  to  Production.  The  revision  of  the  DSARC  decision 
process  making  the  FSD  decision  the  initial  commitment  to  produc¬ 
ing  the  system  means  that  all  data  available  to  support  this 
decision  is  critical.  Historically  the  lack  of  sufficient  and 
comprehensive  T&E  data  to  support  final  baseline  determination 
decisions  has  forced  the  production  of  systems  needing  additional 
design  corrections. 

The  Test  and  Evaluation  process  is  usually  composed  of  four 
major  planning  and  testing  phases  occurring  in  each  acquisition 
phase.  The  first  activity  is  the  development  of  the  Test  and 
Evaluation  Master  Plan  (TEMP),  developed  in  the  Concept  Explora¬ 
tion  Phase.  This  phase's  activities  also  include  initial 
feasibility  testing  in  which  test  experience  from  other,  related 
or  similar  systems  is  evaluated  as  a  source  of  potentially  useful 
insights  in  developing  the  test  plans.  The  T&E  activities  must 
be  designed  to  address  mission-critical  components,  stress 
extremes  in  both  the  mission-related  and  non-mission-related 
system  life  activities,  and  the  reliability  and  maintainability 
of  the  system.  The  latter  is  particularly  important  since  R&M  is 
now  being  incorporated  as  a  contractual  design  requirement  versus 
goal .  This  shift  in  the  specification  of  R&M  thresholds  means 
that  T&E  activities  must  be  designed  to  test  for  the  system's 
designed  R&M  parameters.  This  also  means  that  T&E  planning  must 
be  integrated  with  any  off-line  R&M  technology  maturation 
programs  that  apply  to  the  program. 


i 
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The  Concept  Exploration  Phase  feasibility  studies  focus  on 
both  parts  of  the  system:  the  operational  system  and  the  support 

system.  T&E  activities  relating  to  the  operational  system 
concern : 

•  mission  and  non-mission  stresses, 

•  environmental  testing,  and 

•  system  reliability  (fault  isolation  and  correction). 

Support  system  T&E  efforts  must  be  significantly  more  exten¬ 
sive.  This  is  due  to  the  interrelationships  between  the  develop¬ 
mental  T&E  activities  and  the  development  of  the  support  test 
capability  in  the  fielded  system  (i.e.,  the  diagnostics,  ATE, 

BIT,  etc.).  The  interaction  of  test  results  in  the  development 
of  not  only  the  system  diagnostics  but  the  overall  support  system 
is  critical.  As  discussed  in  the  previous  section  on  hardware 
and  software  design  and  development,  the  critical  shortage  of 
qualified  manpower  available  to  operate  and  maintain  the  system, 
in  conjunction  with  highly  automated  operational  systems,  has 
produced  a  greater  emphasis  on  the  use  of  ATE,  BIT  and  recently 
the  introduction  of  the  MATE  concept  (modular  automated  test 
equipment).  Particular  aspects  of  the  diagnostic  system  are 
discussed  later  in  this  chapter. 

Although  testing  activities  begin  in  the  CE  Phase,  they  are 
concentrated  in  the  later  phases.  The  actual  T&E  activities 
follow  these  Concept  Exploration  Phase  activities.  There  are  two 
categories  of  testing  for  most  systems: 
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•  Developmental  Test  and  Evaluation  (DT&E)  is  conducted 
in  the  Demonstration  and  Validation  Phase  (DT&E-I)  and 
in  the  Full-Scale  Development  Phase  (DT&E-II).  These 
tests  are  conducted  by  the  AFSC  program  office  Test  and 
Evaluation  Directorate,  and  the  contractor  ( s ) ,  -with  the 
support  of  the  Operational  Test  and  Evaluation  (OT&E) 
team,  for  major  systems,  usually  the  Air  Force  Opera¬ 
tional  Test  and  Evaluation  Center  (AFOTEC).  The  major 
objectives  of  DT&E  are  to: 

assess  system  specification  compliance,  deficien¬ 
cies,.  compatibility,  OT&E  readiness,  configuration 
changes ; 

assess  program  risks/trade-offs; 

assess  logistics  supportability/survivability ; 

verify  technical  order  completeness; 

gather  training  program  and  environmental  impact 
data ;  and 

determine  system  performance  limitations. 

•  Operational  Test  and  Evaluation  (OT&E)  is  also  divided 
into  two  parts:  Initial  OT&E  (xOT&E)  and  Follow-on  OT&E 
(FOT&E).  IOT&E  is  conducted  tnroughout  the  first  three 
acquisition  phases,  up  to  the  final  production 
decision.  FOT&E  is  performed  during  the  Production 
Phase.  IOT&E  is  usually  performed  on  prototypes  of  the 
system  -to  test  subsystem  interactions  and  performance, 
and  to  test  the  overall  system's  operational  effective¬ 
ness.  FOT&E,  initiated  after  production  begins  and 
sometimes  continuing  throughout  the  system's  life 
cycle,  can  be  divided  info  two  phases:  FOT&E(I) 
intended  to  refine  operational  suitability  and  effec¬ 
tiveness  estimates  and  to  evaluate  corrective  actions 
resulting  from  IOT&E.  FOT&E(II)  measures  the  system's 
ability  to  meet  changing  operational  requirements, 
refines  operational  tactics  and  programs,  and  identi¬ 
fies  and  confirms  correction  of  system  deficiencies. 

The  major  objectives  of  OT&E  are  to: 

estimate  operational  effectiveness/suitability; 

-  identify  operational  deficiences; 

recommend/evaluate  configuration  changes; 

provide  logistics  support,  training,  tactics, 
operation  and  support  cost  data; 
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determine  technical  data/support  equipment 
adequacy;  and 

estimate  system  survivability. 

Exhibit  4-15  illustrates  the  basic  relationship  of  DT&E  and 
OT&E  in  systems  acquisition.  In  order  to  maximize  the  effective¬ 
ness  of  these  testing  and  evaluation  efforts,  planners  are 
encouraged  to  conduct  them  with  a  view  towards  interchanging  test 
results.  Program  experience  has  shown  that  overlapping  DT&E  and 
OT&E  can  be  beneficial  to  accomplishing  the  program  objectives, 
however,  there  are  inherent  risks  which  must  be  recognized. 

These  are  discussed  in  Chapter  5. 

Software  also  is  subject  to  a  T&E  process  similar  to  hard¬ 
ware.  As  addressed  in  the  previous  discussion,  problems  found 
late  in  a  software  development  effort  tend  to  have  a  more  pro¬ 
found  effect  on  overall  accomplishment  of  the  program  goals  than 
problems  identified  and  corrected  earlier.  Software  designers 
have  tended  to  argue  that  the  critical  function  in  having  an  ade¬ 
quate  and  effective  test  program  is  early  planning  and  develop¬ 
ment  of  a  comprehensive: 

•  test  plan, 

•  test  procedure,  and 

•  test  report. 

The  follow ’ ng  suggested  approach  has  been  developed  and  applied 
successfully  in  a  variety  of  aerospace  software  development 
efforts.-/ 

V  This  is  a  variation  on  the  basic  Test  Plan  format  specified  in 
DoD  Standard  7935,  Automated  Data  Systems  Documentation  Stan— 

has  been  extracted  from  the  unpublished  paper:  Insley, 
.Robert  R. ,  "General  Form  for  a  Verification  Test  Plan  and  Proce¬ 
dure,"  San  Pedro,  CA.  (undated). 
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Exhibit  4-15.  TEST  AND  EVALUATION  CYCLE 


GENERAL  FORM  FOR  A  VERIFICATION 
TEST  PLAN  AND  PROCEDURE 


The  following  is  a  cook  book  procedure  for  writing  a  soft¬ 
ware  verification  plan  and  procedure.  It  is  intended  that  the 
document  will  be  written  in  three  parts.  The  plan  should  be 
written  first  and  submitted  for  review  and  approval,  then  the 
procedure  should  be  written.  After  the  precedure  has  been  run 
and  the  results  reviewed  the  test  report  should  be  written.  This 
approach  is  being  suggested  to  achieve  the  following  conditions: 

•  test  repeatability, 

•  control  of  test  environment, 

•  requirements  traceability,  and 

•  test  conclusiveness. 


A.  TEST  PLAN 

•  PREFACE  -  State  the  software  involved.  This  may  be  a 
module  within  a  system,  a  partially  integrated  system, 
or  a  total  system.  Reference  the  system's  identifica¬ 
tion  and  identify  the  hardware  on  which  the  test  will 
be  run. 

•  1.0.  PURPOSE  -  State  the  purpose  of  the  test  (e.g., 
verify  the  software  requirements  as  they  pertain  to  a 
specific  module,  etc.).  If  the  requirements  are  a  sub¬ 
set  of  a  total  system,  define  the  subset  (e.g.,  display 
requirements,  etc.). 

•  1.1.  GENERAL  DESCRIPTION  -  Describe  how  the  test  is  to 
be  run  (e.g. ,  as  a  stand  alone  test  using  test  drivers 
and  simulated  data,  or  as  a  complete  system  with  live 
data,  etc. ) . 

•  1.2.  REFERENCES  —  List  all  reference  documents  by  ID, 
name  and  revision  (e.g..  Government  standards,  company 
standards,  system  requirements,  etc.).  It  is  assumed 
that  a  Software  Test  Control  standard  is  in  place  and 
will  be  referenced. 
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•  1.3.  TEST  ENVIRONMENT  —  Give  a  general  description  of 
the  hardware  and  software  to  be  used  showing  overview 
diagrams. 

•  1.4.  TEST  OBJECTIVES  -  This  section  should  reference  a 
set  of  appendicies,  consisting  of  the  following  infor¬ 
mation: 

APPENDIX  A  -  Contains  an  abstract  of  requirements 
that  pertain  to  this  test. 

APPENDIX  B  —  Contains  the  test  objectives  which 
were  derived  from  the  requirements  in  Appendix  A. 


B.  TEST  PROCEDURE 


•  2.0.  TEST  PROCEDURE  -  It  is  intended  that  after 
approval  of  the  plan,  the  procedural  portion  will  be 
created  and  added  to  the  document. 

•  2.1.  TEST  DESCRIPTION  -  Define  the  makeup  of  the  test 
in  terms  of  how  many  phases  there  will  be  and  what  each 
phase  will  test.  Also  explain  whether  the  test  will  be 
run  by  an  operator  from  a  terminal  or  automatically  run 
via  command  files  under  the  control  of  the  operating 
system.  If  there  are  unique  characteristics  about  the 
test  which  are  important,  explain  them  (e.g.,  the  test 
will  be  run  using  partially  completed  system  hardware). 

•  2.2.  VERIFICATION  CROSS  REFERENCE  -  This  section 
should  reference  an  appendix  (Appendix  C)  which  will 
contain  a  set  of  matricies  defining  which  objectives 
will  be  tested  in  what  phases  and  in  what  test  steps. 

•  2.3.  HARDWARE  ENVIRONMENT  -  Describe  the  hardware  to 
be  used  noting  equipment  nomenclature.  Also  provide 
detailed  interface  hookup  diagrams. 

•  2.4.  SOFTWARE  ENVIRONMENT  -  Describe  the  support  soft¬ 
ware  to  be  used  (e.g.,  operation  system,  simulation 
program,  etc.).  Also  give  software  ID  and  revision 
data.  In  addition,  the  target  software  should  be 
described.  If  there  are  any  test  data  files  or  command 
files  involved  they  should  be  defined  here. 

•  2.5.  SCENARIO  -  This  section  should  contain  an  actual 
step-by-step  procedure  which  will  explain  just  how  each 
previously  defined  test  objective  is  to  be  tested.  A 
basic  format  will  be  described  here  which  uses  the  test 
step  approach.  There  are  other  formats  which  can  also 
be  used.  The  format  will  be  defined  using  an  inden¬ 
tured  paragraph  numbering  approach. 
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TEST  NAME 


l.X.  -  PHASE  NAME  -  (where  X  is  1  to  the  total 
number  of  phases).  Any  special  test  conditions 
for  the  phase  should  be  described  here. 

l.X.Y.  -  TEST  STEP  NAME  AND  OBJECTIVES )  -  (where 
Y  is  1  to  the  total  number  of  steps).  The  test 
objective(s)  may  be  referenced  by  objective  number 
to  Appendix  B. 

l.X.Y.l.  -  REQUIRED  INPUTS  -  Define  the  inputs  in 
terms,  of  data  flags  to  be  set,  etc.  These  inputs 
must  be  defined  in  terms  of  their  official  names 
(e.g.r  the  associated  mnemonic  as  it  appears  in 
the  code ) . 

l.X.Y. 2.  —  EXPECTED  RESULTS  —  Explicitly  define 
the  expected  results  in  terms  of  values  and  loca¬ 
tions,  displayed  conditions,  etc. 

l.X.Y. 3.  —  INITIATING  CONDITION  —  Give  the  precise 
command ( s )  to  cause  the  step  execution.  If  the 
procedure  is  in  a  command  file  the  command(s) 
should  be  executable  by  that  command  file.  If  the 
command  or  command  sequence  is  to  be  executed 
manually,  then  the  exact  sequence  must  be  defined. 
This  is  important  for  test  repeatability. 

1. X.Y. 4.  —  OBTAIN  RESULTS  —  Give  the  precise  com— 
mand(s)  to  retrieve  the  results.  If  a  command 
file  is  being  used,  then  the  specific  executable 
command ( s )  should  be  defined  which  will  display 
the  contents  of  specific  memory  locations,  dumps, 
etc.  If  the  results  are  to  be  retrieved  manually, 
then  instructions  should  be  given  as  to  how  to 
achieve  them. 

2.0.  -  POST  TEST  ANALYSIS 

2. X.  -  There  should  be  a  tabulation  of  results  for 
each  test  phase  (X).  This  tabulation  should  state 
whether  each  step  has  passed  or  failed.  If  a  step 
has  failed  the  conditions  of  failure  must  be 
stated.  It  is  assumed  that  these  results  will  be 
submitted  to  a  Test  Review  Authority  for  statusing. 


C.  TEST  REPORT 

•  3.0.  TEST  REPORT  -  This  is  the  third  part  of  the  docu¬ 

ment  to  be  created.  It  is  assumed  that  it  will  be 
written  after  the  Test  Review  Authority  has  stated  its 
findings.  The  contents  of  the  report  should  include  at 
least  the  following: 
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•  the  names  of  test  participants  and  observers, 

•  a  brief  description  of  the  test  activity, 

•  the  outstanding  results  (i.e.,  failures), 

•  the  findings  of  the  Test  Review  Authority, 

•  the  justification  of  any  waivers  or  disclaimers,  and 

•  the  actual  copy  of  the  plan/procedure,  test  results  and 
post-test  analysis . 

i 

The  intention  of  this  part  of  this  chapter  has  been  to  con¬ 
sider  some  of  the  readiness-related  aspects  of  hardware  and  soft¬ 
ware  Test  and  Evaluation.  Clearly,  there  are  many  other  aspects 
of  T&E  which  can  impact  system  readiness.  The  reader  is  directed 
to  consult  the  references  following  this  chapter  for  more  detailed 
discussions  on  T&E  as  a  readiness  driver. 


INTEGRATED  LOGISTIC  SUPPORT 


No  discussion  of  materiel  readiness  would  oe  complete 
without  a  discussion  of  Integrated  Logistic  Support  (ILS).  ILS 
and  the  related  Logistic  Support  Analysis  (LSA)  relate  to  a  set 
of  activities  concerned  with  the  analysis  of  requirements  and 
development  of  a  capability  to  maintain  the  system  once  fielded. 
DoDD  5000.39  gives  these  definitions  of  ILS  and  LSA: 

•  Integrated  Logistic  Support.  A  disciplined,  unified, 
and  iterative  approach  to  the  management  and  technical 
activities  necessary  to: 

a.  Integrate  support  considerations  into  system  and 
equipment  design. 

b.  Develop  support  requirements  that  are  related 
consistently  to  readiness  objectives,  to  design, 
and  to  each  other. 
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c.  Acquire  the  required  support. 

d.  Provide  the  required  support  during  the  opera¬ 
tional  phase  at  minimum  cost. 

•  Logistic  Support  Analysis.  The  selective  application 

of  scientific  and  engineering  efforts  undertaken  during 
the  acquisition  process,  as  part  of  the  systems  engin¬ 
eering  process,  to  assist  in: 

a.  Causing  support  consideration  to  influence  design. 

k*  Defining  support  requirements  that  are  related 
optimally  to  design  and  to  each  other. 

c.  Acquiring  the  required  support. 

d.  Providing  the  required  support  during  the  opera¬ 
tional  phase  at  minimum  cost. 

That  same  directive  identifies  10  ILS  elements  and  defines 
them  as  follows: 


Maintenance  Planning.  The  process  conducted  to 
evolve  and  establish  maintenance  concepts  and 
requirements  for  the  lifetime  of  a  materiel 
system. 

Manpower  and  Personnel.  The  identification  and 
acquisition  of  military  and  civilian  personnel 
with  the  skills  and  grades  required  to  operate  and 
support  a  materiel  system  over  its  lifetime  at 
peacetime  and  wartime  rates. 

Supply  Support.  All  management  actions,  proce¬ 
dures.  and  techniques  used  to  determine  require¬ 
ments  to  acquire,  catalog,  receive,  store, 
transfer,  issue,  and  dispose  of  secondary  items. 
This  includes  provisioning  for  initial  support  as 
well  as  replenishment  supply  support. 

Support  Equipment.  All  equipment  (mobile  or 
fixed)  required  to  support  the  operation  and  main¬ 
tenance  of  a  materiel  system.  This  includes 
associated  multiuse  end  items,  ground-handling  and 
maintenance  equipment,  tools,  metrology  and  cali¬ 
bration  equipment,  test  equipment,  and  automatic 
test  equipment.  It  includes  the  acquisition  of 
logistics  support  for  the  support  and  test  equip¬ 
ment  itself. 


4-50 


e*  Technical  Data.  Recorded  information  regardless 
of  form  or  character  (such  as  manuals  and  draw¬ 
ings)  of  a  scientific  dr  technical  nature.  Com¬ 
puter  programs  and  related  software  are  not 
technical  data;  documentation  of  computer  programs 
and  related  software  are.  Also  excluded  are 
financial  data  or  other  information  related  to 
contract  administration. 

f.  Training  and  Training  Support.  The  processes, 
procedures,  techniques,  training  devices,  and 
equipment  used  to  train  civilian  and  active  duty 
and  reserve  military  personnel  to  operate  and 
support  a  materiel  system.  This  includes 
individual  and  crew  training;  new  equipment  train¬ 
ing;  initial,  formal,  and  on-the-job  training;  and 
logistic  support  planning  for  training  equipment 
and  training  device  acquisitions  and  installations. 

g.  Computer  Resources  Support.  The  facilities, 
hardware ,  software ,  documentation ,  manpower,  and 
personnel  needed  to  operate  and  support  embedded 
computer  systems. 

h.  Facilities.  The  permanent  or  semipermanent  real 
property  assest  required  to  support  the  materiel 
system,  including  conducting  studies  to  define 
types  of  facilities  or  facility  improvements, 
locations,  space  needs,  environmental  require¬ 
ments,  and  equipment. 

i •  Packaging,  Handling,  Storage,  and  Transportation. 
The  resources,  processes,  procedures,  design 

consideratinos,  and  methods  to  ensure  that  all 
system,  equipment,  and  support  items  are 
preserved,  packaged,  handled,  and  transported 
properly,  including  environmental  considerations, 
equipment  preservation  requirements  for  short-  and 
long-term  storage,  and  transportability. 

j.  Design  Interface.  The  relationship  of  logistics- 
related  design  parameters,  such  as  R&M,  to 
readiness  and  support  resource  requirements. 

These  logistics— related  design  parameters  are 
expressed  in  operational  terms  rather  than  as 
inherent  values  and  specifically  relate  to  system 
readiness  objectives  and  support  costs  of  the 
materiel  system. 

The  variety  of  elements  included  under  the  umbrella  of  ILS  gives 
an  indication  of  the  multifaceted  nature  of  support.  Emphasis  on 
readiness  must  mean  emphasis  on  ILS. 
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Historically,  ILS  has  been  given  less  concern  and  a  lower 
priority  in  systems  acquisition.  In  trade-offs  between  perfor¬ 
mance,  acquisition  costs,  schedule  constraints,  and  support, 
support  has  had  a  tendency  to  come  out  the  loser.  Expenditure  of 
near-term  funds  on  immediate  problems,  primarily  in  technical 
design,  meant  that  ILS  planning  and  acquisition  has  frequently 
been  deferred.  The  realization  that  having  a  mission— capable 
system  means  having  an  in-place  support  system  has  been  known  to 
the  user  community  and  logisticians  (or  logistically-sensitive 
designers)  for  a  long  time.  However,  this  point  has  been  fre¬ 
quently  missed  by  decision-makers. 

One  way  of  looking  at  ILS  is  to  consider  the  elements  that 
relate  to  developing  an  ILS  capability  and  those  which  must  be  in 
place  in  order  to  have  an  ILS  capability.  The  former  could 
include : 

•  maintenance  planning,  and 

•  design  interfaces. 

In  order  to  actually  have  a  fielded  support  capability,  such  as 
one  sufficient  to  meet  IOC,  the  remaining  elements  of  logistics 
support  must  be  in  place: 

•  adequate  number  of  properly  trained  manpower  available 
upon  system  fielding; 

•  supply  support  (initial  and  replenishment  spares  and 

associated  inventory  management  and  control  procedures); 

•  support  equipment,  including  system  unique  transporta¬ 

tion  and  storge  equipment,  test  equipment  (BIT  and 
ATE),  ground  support  equipment,  tools  (e.g.,  metrology 
and  calibration  equipment),  etc.; 
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•  technical  data/documentation/orders,  including  the 
detailed  description  of  the  system  configuration,  the 
most  recent  as  well  as  all  versions  of  the  system  cur¬ 
rently  in  the  field,  full  documentation  of  the  hardware 
and  software  maintenance  procedures  for  the  mission  and 
support  equipment;  and  all  technical  or  engineering 
drawings,  manuals  or  instructions; 

•  computer  resources  support,  the  Embedded  Computer 
Resources  (ECR)  to  operate  and  support  the  system,  and 
all  of  the  elements  of  that  capability,  to  the  full 
breadth  of  maintenance  support  (i.e*,  below  depot 
level ) ; 

•  facilities,  the  flight  line  maintenance,  avionic  inter¬ 
mediate  support  facilities,  training  facilities,  etc., 
and  the  organization  and  arrangement  of  these  facili¬ 
ties  in  a  manner  promoting  efficiency  ^nd  productivity; 
and 

•  development  of  appropriate  containers  for  the  trans¬ 
portation  and  storage  of  the  system,  components  and 
support  equipment. 

In  planning  and  developing  an  ILS  capability  and  in  conduct¬ 
ing  the  associated  LSA,  the  manager  must  maintain  cognizance  of 
certain  primary  concerns.  First,  in  order  to  maximize  effective¬ 
ness,  as  with  requirements  formulation,  hardware  and  software 
design  and  development,  and  test  and  evaluation,  ILS  activities 
must  be: 

•  initiated  early  (in  Concept  Exploration); 

•  actively  managed  throughout  the  program; 

•  managed  with  a  priority  toward  maintaining  oversight  of 
the  interrelationships  and  dependencies  among  the  ILS 
elements;  and 

•  focused  as  early  as  possible  on  analyzing  requirements 
and  associated  risks  in  producing  an  effective  and 
integrated  support  capability. 

As  discussed  elsewhere  in  this  handbook,  historically,  ILS  ele¬ 
ments  have  been  deferred,  postponed,  truncated,  or  eliminated  in 
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systems  acquisition.  The  result  has  been  the  fielding  of  a  sys¬ 
tem  without  a  sufficient  support  system.  This,  in  many  cases, 
means  buying  a  support  capability  or  parts  thereof  via  interim 
contractor  support.  There  are  pros  and  cons  to  this  approach  in 
that  is  allows  the  system  to  be  fielded  without  a  complete  logis¬ 
tics  capability  residing  in  the  Air  Force.  It  also  allows  for 
continued  maturation  of  the  R&M  and  maintenance  under  the  super¬ 
vision  of  highly  qualified  technicians  and  engineers.  On  the 
other  hand,  this  maturation  may  be  somewhat  artificial  in  that, 
like  OT&E ,  the  true  system  R&M  cannot  be  realistically  evaluated 
because  "Ph.D  engineers  rather  than  E-3s  are  performing  the 
analysis."  There  is  also  the  fact  that  contractor  personnel  are 
not  military  personnel,  and  are  not  part  of  the  deployable  infra¬ 
structure  . 

Another  fundamental  aspect  of  ILS  has  been  alluded  to  in  this 
discussion  and  more  generally  in  the  Chapter  2  discussion  of 
problems  with  managing  to  readiness:  there's  a  lot  that  goes 
into  having  an  integrated  support  system.  The  ten  elements  of 
ILS  are  not  all  of  the  factors  influencing  the  development  of  an 
ILS  capability.  Significant  interactions  exist  among  many 
groups  within  the  PO  or  working  in  cooperation  with  the  PO,  as 
can  be  seen  from  the  following  quote  from  the  System  Engineering 
Management  Guide. 

Although  Logistics  is  primarily  concerned  with  the  system 
deployment  and  operations  phase,  it  is  a  major  cost  driver  and  a 
vital  issue  in  system  readiness,  reliability,  sustainability, 
maintenance  concept,  maintenance  level  selection,  human  engineer¬ 
ing,  _  safety,  testability,  repair/replace  decisions,  special 
training,  standarizat ion ,  etc.  Most  of  the  significant  decisions 
that  affect  logistic  support  cost  will  have  been  made  before  the 
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production  phase  begins.  On  a  typical  program,  95  percent  of  the 
total  program  costs  will  have  been  committed  by  the  Critical 
Design  Review  ( CDR )  milestone  in  the  FSD  phase.  Logistics  con¬ 
siderations  have  far  reaching  effects  on  inventory,  manpower, 
tools,  and  equipment." 

ILS  task  activities  peak  in  the  FSD  Phase.  It  is  in  this  phase 
that  much  of  the  logistic  system  is  fully  designed  and  developed. 
Exhibit  4—16  shows  the  major  ILS— related  activities  and  their 
relationships,  in  the  FSD  phase.  Much  of  the  planning  and 
initial  development  activities  should  be  conducted  or  at  least 
initiated  long  before  this  phase,  since  delaying  until  FSD  signi¬ 
ficantly  limits  the  impact  any  of  the  ILS  elements  can  have  on 
the  design.  Exhibit  4—17  illustrates  how  decisions  in  the  main¬ 
tenance  concept  influence  and  interact  with  the  ILS  development 
process . 

In  considering  system  readiness  and  ILS,  several  of  the 
capability-related  ILS  elements  play  particularly  critical  roles. 
These  elements,  in  fact,  play  dual  roles  in  that  they  contribute 
to  the  overall  logistics  support  capability,  and  they  influence 
one  of  the  major  systems  that  relates  to  readiness:  the  diagnos¬ 
tic  system.  These  elements  are: 

•  manpower  and  personnel, 

•  training  and  training  support, 

•  support  equipment,  and 

•  computer  resources  support. 

Manpower  and  personnel,  and  training  and  training  support, 
have  for  a  long  time  been  thought  of  collectively  as  manpower, 
personnel  and  training  (MPT).  Efforts  in  the  last  several  years 
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S°urce:  fp^lg^lgaAneering  Management  Guide,  Defense  Systems  Management  College, 
Exhibit  4-16.  INTEGRATED  LOGISTIC  SUPPORT  TASK  FLOW  DURING  FSD 
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Exhibit  4-17 .  MAINTENANCE  TASK  FLOW  AND  INTEGRATION  WITH  ILS 
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have  been  directed  toward  recognizing  that  they  must  be  con¬ 
sidered  separately,  both  in  terms  of  developing  estimates  of 
requirements  and  in  developing  the  capability.  However,  they 
both  relate  to  the  human  element  in  the  logistics  equation  (as 
opposed  to  the  human  factors  element  in  design). 

In  an  effort  to  respond  to  shifting  demographics,  the  criti¬ 
cal  shortage  of  adequately  trained  enlisted  personnel,  with  the 
required  aptitudes,  designers  have  steadily  increased  the  use  of 
automated  diagnostic  equipment  -  BIT,  ATE  and  MATE.  This  trade¬ 
off  has  also  been  intended  to  reduce  the  proliferation  of  support 
test  equipment  in  the  diagnostic  function.  However,  these  trades 
have  not  been  universally  successful.  It  is  useful  to  consider 
these  problems  because  the  diagnostic  capability  is  becoming 
increasingly  the  focus  of  the  embedded  support  capability  of  the 
deployed  system.  An  effective  diagnostic  system  impacts  the 
quantity,  quality  and  variety  of  systems  spares,  personnel, 
support  equipment,  and  the  overall  scope  of  the  support  establish¬ 
ment,  as  well  as  the  confidence  the  operators  and  maintainers 
have  in  the  system. 

The  development  and  testing  of  the  diagnostics  system  is  a 
major  concern  of  AFOTEC.  Their  studies  and  experience  in  opera¬ 
tional  test  and  evaluation  have  shown  some  of  the  problems  in 
trading  off  between  automation  and  manpower.  Many  of  these 
findings  have  also  been  addressed  in  the  IDA/OSD  Reliability  and 
Maintainability  Study.  These  are  briefly  discussed  below. 

— /  This  statement  and  others  concerning  the  state  of  the  art  in 
diagnostics,  has  been  taken  from  the  IDA/OSD  Reliability  and 
Maintainability  Study.  Other,  similar  extractions  are  noted 
where  appropriate. 
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Diagnostics  as  an  area  of  engineering  design,  is  an  imma¬ 
ture  discipline  when  compared  to  reliability.  In  diagnostics, 
there  are  not  accepted  definitions  of  requirements  that  can  be 
used  for  contracting  that  are  directly  understandable  to  a 
designer  and  that  can  be  related  to  field  experience." 

Also,  the  definition  of  what  constitutes  the  diagnostic  system 

can  vary  from  system  to  system. 

Current  systems  for  providing  field  experience  data  are  not 
sufficient  to  support  diagnostic  system  design  efforts  because 
they  do  not  provide  sufficiently  detailed  information  on  the 
system  f ailure-resolution/cannot  duplicate  actions. 

The  use  of  automated  systems  in  diagnostics  has  tended  to 
limit  the  inherent  understanding  of  the  operations  of  the  system, 
making  it  more  difficult  to  maintain  the  diagnostic  equipment, 
and  creating  distrust  in  the  information  provided  by  the  system. 

The  more  complex,  the  relationship  between  the  end  item  and 
the  support  equipment,  the  more  difficult  it  is  to  keep  the  sys¬ 
tems  operational/maintained.  Hybrid,  integrated  diagnostic 
systems,  combining  automation  and  manual  requirements  are  even 
more  difficult  to  analyze  because  of  the  unknowns  associated  with 
the  manual/automated  interactions.  The  manual  aspects  have  much 
uncertainty  and  are  difficult  to  quantify.  Highly  automated 
systems,  however,  cannot  be  readily  fixed  in  the  field.  The 
designer  must  develop  the  appropriate  mix  of  test  equipment 
availability,  reliability  and  support,  personnel  availability  and 
training,  available  technology,  and  life  cycle  cost,  to  develop 

the  most  balanced  system  given  the  user's  requirements  and  opera¬ 
tional  needs. 
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The  use  of  automated  or  semi-automated  diagnostics  means 
that  resources  must  be  dedicated  very  early  in  the  concept  devel¬ 
opment,  particularly  regading  large  scale  integrated  (LSI)  cir¬ 
cuits  or  very  high  speed  integrated  circuits  (VHSIC). 

The  failure  of  BIT  equipment  and  ATE  to  fulfill  the  facility 
isolation  and  detection  requirements,  to  reduce  the  false  alarms/ 
cannot  duplicate  failures,  has  in  many  cases  meant  a  significant 
increase  in  maintenance  personnel  required  for  the  system,  once 
fielded.  The  E-3A  is  a  significant  example,  with  a  new  require¬ 
ment  for  an  additional  18  radar  and  9  support  equipment  techni¬ 
cians,  each  of  which  required  six  additional  months  of  formal 
training  and  25  months  of  on-the-job  training  (OJT),  just  to 
develop  sufficient  proficiency  in  troubleshooting. 

Both  AFOTEC  and  the  IDA/OSD  Reliability  and  Maintainability 

Study  identified  a  significant  problem  with  the  way  in  which 

diagnostic  system  parameters  (e.g.,  failure  rates,  false  alarms, 

faults  isoloated,  etc. )  were  expressed  in  contractual  statements 

of  work.  Targeted  goals  can  be  easily  interpreted  to  provide  a 

bare  minimum  capability.  The  following  quote  from  an  AFOTEC 

briefing  on  weapon  systems  diagnostics  describes  what  can 
9/ 

happen 

. . . .We  have  not  done  a  good  job  in  the  way  we  articulate 
our  needs  to  contractors, ...  and,  we  as  testers  needed  to  improve 
evaluation  methods  to  provide  decision  makers  more  meaningful 
information.  [Lets]  look  at  one  way  specifications  have  been 
interpreted  in  the  past. 

-*■  -1  use  some  number  values  which  were  an  in  some  cases 

still  are  familiar  and  commonly  used  to  specify  automated  needs, 

2/  Lohr,  Major  Douglas,  "Weapon  Systems  Diagnostics,"  (Briefing), 
Air  Force  Operational  Test  and  Evaluation  Center,  (undated). 
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and  design  a  90  percent  fault  detection  and  80  percent  isolation 
system  for  you.  When  we,  the  Air  'Force,  say  90/80  we  want  90 
percent  of  the  faults  discovered  which  can  occur  and  80  percent 
°f  all  faults  isolated.  I,  as  a  contractor,  can  interpret  this 
specification  [in  the  following  manner].  If  I  expect  100  faults 
in  1000  operating  hours  for  my  system,  of  10  black  boxes  and  I 
know  that,  based  on  failure  modes  and  effects  analysis  90  of 
those  100  faults  will  occur  in  5  of  10  boxes,  I  will  only  auto¬ 
mate  5.  I've  fulfilled  the  90%  FD  requirement.  I  can't  be 
expected  to  automatically  isolate  what  I  don't  automatically 
detect.  I'll  automate  only  4  of  5  boxes  to  get  my  80%  isolation 
capability.  You  can  see  the  disconnect.  What  I  wind  up  with  is 
90%  FD  and  72%  FI  going  in  -  the  bare'  minimum.  We  first  noted 
this  type  of  problem  with  the  E-3 . " 

The  following  is  an  approach  developed  by  AFOTEC  for  devel¬ 
oping  a  more  effective  diagnostic  capability,  addressing  also  the 
relationship  of  diagnostics  to  logistics. 

"As  early  as  possible  users  should  develop  and  articulate 
their  concepts.  Next  they  should  determine  constraints,  keeping 
the  repair  process  in  mind.  They  should  set  optimum  bounds  on 
downtime  and  establish  what  will  be  available  in  manning  and 
skills  and  what  support  equipment  they  will  want  or  can  live  with 
considering  deployment  and  home  base  support.  Then  based  on 
constraints  develop  requirements  and  state  emphatically  the  outer 
limit  acceptable  in  order  to  do  the  mission  with  the  money 
available.  And,  continually  work  with  developer  provisioner  and 
tester  to  insure  what  has  typically  happended  doesn't  continue: 
namely,  logistics  lagging  hardware  development  and  operational 
requirements. 

"As  developers,  first  concern  should  be  user  requirements 
once  the  Air  Force  need  is  established,  rather  than  continuing  to 
attempt  to  achieve  100  percent  diagnostics  through  a  contractual 
specification  as  a  separate  effort,  AFOTEC  recommends  a  blend  of 
automated  and  manual  diagnostics  techniques  be  developed  based  on 
user  maintainability  parameters,  such  as  maximum  downtime  or 
turn-around  time.  This  optimum  blend  can  be  achieved  through 
direction  of  trade  studies  in  the  statement  of  work  and  insis- 
tance  on  concurrent  hardware  and  diagnostics  development.  In  this 
way  the  customer  should  get  a  more  usable  product.  Some  of  the 
shortfalls  of  the  past  can  be  avoided  by  working  within  the 
limits  of  current  technology.  Thorough  integrated  logistics 
support  planning  can  mold  diagnostics  into  a  whole  rather  than 
fragmenting  BIT  from  support  elements  such  as  training,  technical 
data  and  support  equipment  over  long  periods  of  time.  Interim 
contractor  support  (ICS)  is  normally  available  for  some  specified 
period  for  each  new  system.  Part  of  that  ICS  could  be  a  require- 
m®nt  to  mature  diagnostics  in  the  field.  [Developers  and  the 


4-61 


* 


V 


test  community]  are  all  too  familiar  with  the  number  of  problems 
that  only  show  up  after  a  period  of  use  in  the  operation  environ¬ 
ment.  One  of  the  most  successful  methods  used  to  fix  initial 
problems  found  during  IOT&E  was  with  the  EF-111A.  On  that  system 

n?  SJ3  W*XlZTS'  softYare  engineers  and  representaives  from  the 
SPO,  tac:  and  the  operational  test  center  worked  together  to  close 
tne  loop  and  plug  holes  in  diagnostics. 


IOC)  percent  coverage  [is  needed],  but  overlap,  particularly 
in  critical  fault  detection  and  isolation  should  be  considered. 
For  instance  a  manual  system  and  additional  formal  training  to 

r°h  •  vt"  °f  an  °Perator  reported  system  malfunction 
[which]  should  [be  able  to  be  found]  with  maintenance  BIT,  but 
may  not  be  able  to  [be  found] . 


And  finally,  a  good  way  to  mature  diagnostics  early  is  to 
require  the  contractor  to  use  technical  data,  BIT  and  [support 
equipment]  as  blue  suiters  will,  and  have  everyone  use  existinq 
support  during  testing.  ^ 


[AFOTEC  also  has  suggestions  regarding  the  role  of  the  test 
community.]  "The  earlier  [test  groups]  get  involved,  the  better. 
L They ]  need  to  build  [their]  approaches  and  plans  based  on  user 
nee  s,  which  makes  it  mandatory  that  [they]  work  with  users  and 
developers  to  get  the  criteria  for  the  systems.  [The  test  com¬ 
munity  is]  now  folding  diagnostics  into  the  areas  where  [they] 
feel lt  bel°ngs;  maintainability  as  it  aids  technicians  in  their 
troubleshooting  efforts;  mission  reliability  and  how  well  BIT 
informs  the  operator  of  the  condition  of  a  system,  particularly, 
regarding  critical  faults;  and  the  elements  of  logistics  support 
which  are  interdependent  when  considering  diagnostics.  Finally, 
since  diagnostics  need  to  be  systematically  matured,  [testers] 
need  to  track  and  test  them  through  their  life  cycle." 


While  the  preceeding  discussion  could  not  possibly  address 
all  of  the  potential  logistics  activities  related  to  materiel 
readiness,  it  has.  hopefully,  given  a  different  perspective  to 
considering  the  relationships.  For  a  more  detailed  analysis  of 
the  activities  the  program  planner  and  decision  maker  should 


consider  in  logistics  planning,  a  key  reference  to  consult  is 
MIL-STD-1388-1A,  Logistic  Support  Analysis  (11  April  1983).  This 
standard  identified  the  key  analyses  that  must  be  performed  in  a 


comprehensive  LSA  and  provides  the  basis  for  tailoring  an  ILS 
acquisition  strategy.  Extracts  from  this  standard  are  in 


Appendix  E. 
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In  addition  to  these  critical  technical  functional  areas, 
another  area  of  activity  plays  a  key  role  in  developing  materiel 
readiness:  Program  Structuring  and  Management.  This  is  the 

final  functional  area  discussed  in  this  chapter. 

PROGRAM  STRUCTURING  AND  MANAGEMENT 

In  designing  a  system  emphasising  the  ultimate  readiness 
capability  of  the  system,  the  Program  Manager  must  recognize  that 
certain  generic  factors  will  influence  how  best  to  set  the 
program  priorities.  Among  these  factors  are: 

•  variations  in  programs  and  acquisition  environment, 

•  interrelationships  and  dependencies  of  program 
elements , 

•  concurrency,  and 

1°  / 

•  scheduling.  ' 

The  first  of  these  factors  is  discussed  below.  The  other  factors 
are  discussed  elsewhere  in  this  handbook. 

There  are  four  main  areas  that  come  under  the  first  factor 
mentioned  above: 

•  type  of  system, 

•  expected  operational  environment  and  usage, 

•  technology  aspects  of  the  system,  and 

•  the  current  acquisition  environment. 


IQ/tVip  substance  of  this  discussion  is  based  on  information  and 
ideas  contained  in  the  IDA/OSD  Reliability  and  Maintainability 
Study,  Volume  III,  Case  Study  Analysis,  Institute  for  Defense 
Analyses,  November  1983.  Unless  otherwise  stated,  all  quotations 
are  from  that  volume  of  the  study. 
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1 •  Type  of  System 

The  type  of  system  (i.e.,  electronic  or  mechanical) 
being  acquired  will  significantly  infuence  the  parameters  which 
are  used  to  define  the  required  R&M/ readiness  capability  of  the 
system,  since  some  elements,  goals  or  requirements  are  more 
applicable  to  some  types  of  systems  than  others.  Examples  would 
be  thermal  analysis  applied  to  engines  and  radars;  derating 
criteria  in  electronics;  and  margins  of  safety  or  allowable 
material  strengths  applied  to  engines  and  airframes. 

2 •  Operational  Environment 

The  operational  environment  also  influences  the  system 
R&M,  since  the  same  system,  subsystem  or  component  may  vary 
significantly  in  reliability  depending  on  the  system  in  which  it 
is  installed  and  how  that  system  is  used.  This  means  that 
"...there  is  inherent  uncertainty  in  attempting  to  translate 
field  occurrences  to  design  features/attributes  of  one  program 
for  use  in  planning  another  program  without  first  attempting  to 
understand  the  many  subtle,  but  potentially  major,  impactors." 

Variation  in  the  placement  of  a  system  on  different 
Platf°rms  may  also  have  a  significant  influence  on  reliability- 
related  design  features,  and  in  the  ultimate  reliability  of  the 
system.  An  example  is  the  20mm  Gatling-type  M61-A1  gun, 
installed  in  the  F-15,  F-4E,  F-16  and  F/A-18.  This  gun  is 
installed  in  a  different  position  in  each  of  these  aircraft,  and 
as  a  result  is  subject  to  different  forces  and  stresses.  The 
potential  affects  of  different  configurations  must  be  examined 
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not  only  in  developing  the  initial  analysis  of  the  planned  opera¬ 
tional  analysis,  but  also  in  integrating  the  gun  characteristics 
and  reliability  capabilities  in  the  overall  aircraft  design. 

In  addition  to  these  variations,  it  is  also  necessary 
to  consider  the  different  operational  environments  for  missions, 
particularly  in  joint  programs.  Air  Force  systems  may  be 
deployed  virtually  anywhere  and  the  same  basic  system  may  have  to 
operate  under  extremely  different  climatic  and  environmental  con¬ 
ditions.  Jointly  developed  systems,  those  developed  in  conjunc¬ 
tion  with  the  other  Services,  may  compound  this  problem  due  to 
additional  requirements  to  operate  in  different  environments 
(e.g.,  salt  and  sea  for  Navy  systems,  and  mud  and  sand  for  Army 
systems . ) 

3 .  Technology 

"The  evolutionary  stage  of  the  hardware/software 
involved  and  the  degree  of  the  state-of-the-art  design  in  a  sys¬ 
tem  play  a  major  role  in  attempting  to  structure  R&M  elements  in 
a  program. "  Analyses  of  the  F-15  and  F-16  radar  systems  development 
histories  shows  that  the  multi-stages,  evolutionary  development 
of  the  system  allows  for  steadily  increased  reliability.  These 
radars  evolved  through  the  adaptation  of  state-of-the-art  tech¬ 
nological  advances  in  solid  state  circuitry,  micro— processor 
development  and  the  enhanced  capabilities  available  to  transfer 
previously  mechanical/electronic  functions  to  software  (i.e., 
programmable  signal  processors  (PSP)). 


Current  Acquisition  Environment 


* 
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The  current  acquisition  environment,  as  evidenced  in 
the  Acquisition  Improvement  Program  (AIP),  DoDD  5000.1,  -DoDD 
5000.39,  and  MIL-STD-1388-1A,  emphasizes  increased  concern  on 
system  supportability  and  readiness  in  system  design.  In  addi¬ 
tion  to  these,  there  are  a  number  of  directives,  instructions, 
standards,  guides  and  studies  which  address  particular  aspects  of 
the  readiness/supportability  picture.  Many  of  these  documents 
emphasize  the  need  to: 

•  be  aware  of  the  multiple  factors  and  interdependencies 
which  relate  to  system  readiness; 

•  plan  for  managing  these  interdependencies  in  a  way  that 
recognizes  that  coordination,  communication,  and  inten¬ 
sive  monitoring  are  necessary  to  comprehensively  manage 
to  maximize  readiness; 

•  provide  resources  (personnel,  funding,  time)  to  allow 
for  the  effective  and  adequate  consideration  of  readi¬ 
ness  drivers;  and 

•  begin  planning  and  analysis  of  readiness— related 
requirements  early  in  the  acquisition;  in  Concept 
Exploration,  and  continue  these  efforts  throughout  the 
acquisition,  analyzing  field  results  after  deployment. 

Funding  is  a  particular  concern  in  maintaining  an 
effective  readiness-oriented  program.  Exhibit  4-18  shows  the 
suggested  funding  profile  for  a  program  with  the  general  trend 
which  presently  occurs  in  actual  programs,  by  design  phase.  In 
programs  funded  for  performing  the  necessary  readiness-related 
activities  early  in  the  acquisition  process,  such  as  tradeoffs 
between  R&M  and  performance,  the  funding  curve  would  show  a  pat¬ 
tern  with  increased  spending  in  the  earlier  phases.  As  shown  in 
Exhibit  4-18A,  what  presently  tends  to  occur  is  a  much  shallower 
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Exhibit  4-18d. 
F/A-18  DEVELOPMENT 
COST  PROFILE 


Source:  IDA/OSD  Reliability  and  Maintainability  Study,  Vol.  Ill,  Case  Study  Analysis 

Exhibit  4-18.  SUGGESTED  VERSUS  ACTUAL  FUNDING  PROFILES 


funding  curve,  with  significant  emphasis  on  the  late  FSD  Phase 
activities  and  early  Production  Phase.  Such  emphasis  means  that 
cost  benefits  accrued  by  early  identification  and  analysis  of 
high  risk  elements  may  not  occur.  While  these  risks  may  ulti¬ 
mately  be  resolved,  the  resolution  may  be  accomplished  much  later 
in  the  pogram  and,  therefore,  at  a  much  higher  cost.  In  terms  of 
fixing  problems  and  reducing  risks,  time  does  mean  money. 

Emphasizing  front-end  analysis  also  means  considering 
approaches  for  maturing  the  design  and  technology  earlier  in  the 

program.  The  following  quote  explain  why  this  is  seen  as  a  use¬ 
ful  approach. 


mus 


For  programs  with  severe  budget  constraints,  consideration 
t  be  given  to  focusing  on  front-end  design  activities  to 
deveiop  high  design  potential  and  then  planning  for  an  extended 
R&M  growth/maturation  program,  using  data  obtained  from  produc¬ 
tion  and  field  usage  to  identify  problems  and  establish  design 
fixes  ... .starting  a  growth  program  with  a  more  mature  design  and 
growing  the  diagnostics,  simultaneously,  is  a  promising  approach 
favored  by  many  experts .... 

In  the  past  it  is  not  uncommon  to  begin  growth  testinq  with 
an  immature  design.  Test  time  was  wasted  identifying  the  major 
faults  that  could  have  been  eliminated  with  a  stronger  up-front 
effort.  This  up-front  effort  would  facilitate  more  efficient  use 
o.  test  resources  to  identify  the  latest  problems  that  desiqn 
analysis  will  not  detect ... .With  the  present  funding  profile,  the 
designer  s  innovative  and  creative  thinking  can  be  severly  con- 
strained  by  schedule.  His  primary  emphasis  is  to  put  a  system 
together  that  functions  with  a  minimum  of  effort  and  to  develop  a 
design  which  not  only  functions  but  also  does  so  with  a  low 
failure  rate.  This  constraint  currently  forces  the  designer  or 
program  manager  to  delay  the  development  of  a  diagnostic  system 
until  well  into  the  testing  phase,  a  major  problem  identified  in 
[several]  case  studies." 


Another  element  of  the  current  acquisition  environment 
is  the  set  of  current  acquisition  policies.  A  major  driver  of 
the  system's  ultimate  readiness  capability  when  fielded  is  the 
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reigning  acquisition  policy  regarding  overlapping  of  start  of 
production  and  completion  of  the  FSD  Phase  testing.  The  previous 
policy  of  "fly  before  buy"  minimized  the  use  of  concurrency  in 
these  phases  and  instead  emphasized  completion  of  development  and 
final  Operational  Testing  II  before  initiation  of  full  produc¬ 
tion.  This  allowed  adequate  time  to  develop  the  system,  test  it, 
and  fix  problems  before  production.  The  Army's  T-700  Engine, 
used  in  the  AH-64  Apache  and  UH-60A  Black  Hawk  helicopters,  is  an 
example  of  such  a  program.  While  the  engine  and  the  user  air- 
craft  had  concurrent  FSD  Phases,  much  development  work  occurred 
in  the  early  engine  development  program.  The  engine  was  devel¬ 
oped  over  a  12  year  period  and  the  adequate  funding  of  this 
development  process  is  seen  as  a  major  reason  for  the  engine's 
success . 

In  order  to  allow  for  adequate  development  of  a  readi¬ 
ness  capability,  concurrent  programs  (examples  of  which  are  the 
F-15,  F-16  and  F/A-18)  must  be  structured  so  that: 

•  R&M  activities  are  implemented  early  in  the  program 
(i.e.,  Concept  Exploration  Phase)  and  given  emphasis 
throughout  the  program, 

•  R&M  design  disciplines  are  enforced  up-front,  and 

•  an  R&M  growth  program  is  implemented  to  extend  through 
the  FSD  Phase  into  production. 
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CHAPTER  5.  CONCURRENCY-RELATED  ACTIVITIES 


Concurrency  can  be  used  in  a  variety  of  different  applica¬ 
tions  within  a  program.  As  discussed  in  Chapter  2,  the  term 
itself  can  be  interpreted  to  mean: 

•  parallel  (back-up)  technological  development; 

{ 

•  simultaneous,  but  independent,  subsystem  development 

and  testing; 

•  co-production;  and 

•  overlap  of  dependent,  normally  sequential  activities. 

The  last  interpretation  is  the  one  applied  in  this  handbook.  It 
can  also  be  applied  to  different  magnitudes  within  the  program, 
(determined  by  how  much  the  activities  overlap),  and  at  different 
levels  of  tasking  -  from  overlapping  acquisition  phases,  to  over¬ 
lapping  tasks  associated  with  level  six  activities  in  a  WBS. 

This  chapter  focuses  on  the  major  kinds  of  materiel  readi¬ 
ness-related  activities  which  tend  to  be  concurrently  scheduled. 
Some  of  these  activities  have  been  discussed  in  light  of  their 
concurrency  considerations  in  the  preceeding  chapter.  The 
activities  discussed  in  this  chapter  are  only  some  of  the  total 
number  that  can  be  concurrently  scheduled.  However,  they  are 
activities  which,  if  not  closely  managed,  can  cause  significant 
impacts  on  readiness.  The  reader  should  use  this  discussion  as  a 
starting  point  from  which  to  examine  materiel  readiness  implica¬ 
tions  of  his  or  her  particular  concurrency  decisions. 

Three  major  activities  are  discussed  in  this  chapter: 


I 
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SPECIFICATION  DEVELOPMENT  AND  DESIGN  REVIEWS,  as  criti¬ 
cal  activities  in  the  Requirements  Formulation  and  the 
design  and  development  of  the  system? 

FULL-SCALE  DEVELOPMENT  TESTING  AND  PRODUCTION,  the  form 
of  concurrency  most  frequently  discussed;  and 


ILS  ELEMENT  DEVELOPMENT,  particularly  the 
of  the  maintenance  facility. 


development 


These  activities  are  discussed  in  terms  of  the  potential  impacts 
of  concurrently  scheduling  them,  not  how  they  should  be  sched¬ 
uled.  The  next  part  of  the  handbook,  comprised  of  Chapters  6,  7 

and  8,  examines  that  concern. 

‘  r 

-  Y  .y 

SPECIFICATION  DEVELOPMENT  AND  DESIGN  REVIEWS 


In  Chapter  4,  the  discussion  of  the  requirements  formulation 
process  included  brief  descriptions  of  the  major  elements  of 
developing  requirements  documents: 

•  the  development  of  specifications  documenting  the 
system  requirements  and  design  characteristics, 

•  the  maintenance  of  design  baselines  as  a  mechanism  for 
configuration  management  and  control,  and 

•  the  conduct  of  periodic  reviews  of  the  hardware  and 
software  design  documentation. 

The  specific  schedule  for  developing  and  reviewing  these  is  left 
to  the  Program  Manager.  In  terms  of  concurrency,  this  means  that 
the  overlapping  of  design  activities  and  the  conduct  of  the 

reviews  can  produce  significant  problems  in  configuration  manage- 
men  t . 


A  product  of  reviewing  the  design  is  frequently  a  need  to 
conduct  further  analysis  of  some  aspect  of  the  design.  This  is 
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immature  or 


most  frequently  the  case  in  programs  applying  new, 
developing  technologies,  where  the  technological  risks  are  much 
higher.  The  configuration  is  in  a  state  of  flux  in  that: 

•  portions  of  systems  may  be  under  revision,  and  there 
may  be  numerous  revisions  of  some  parts  of  the  system, 
while  other  parts  are  fairly  stable,  more  mature,  and 
not  representing  as  m,any  unknowns; 

•  the  capability  to  communicate  the  status  of  the  various 
interdependent  activities  is  taxed  if  there  is  not  a 
management  information  system  sufficiently  responsive 
to  the  need,  already  in  place; 

•  the  jurisdictional  conflicts  between  different  func¬ 
tional  areas  that  occur  due  to  unclear  delineation  of 
responsibilities  results  in  confusion  as  to  the  inter¬ 
pretation  and  ramifications  of  the  status  of  critical 
configuration  items. 

The  ability  to  schedule  review  of  the  specifications  not  at 
the  end  of  the  phase  in  which  they  are  completed,  but  in  the 
beginning  of  the  succeeding  phase  means  that  contractual  vehicles 
may  be  developed  that  are  based  on  designs  that  may  change. 


FULL-SCALE  DEVELOPMENT  TESTING  AND  PRODUCTION 


By  far,  the  most  critical  and  highly  visible  of  the  activi¬ 
ties  that  can  be  concurrently  scheduled  are  the  test  and  evalua¬ 
tion  (T&E)  activities  and  the  production  design  decisions. 

Exhibit  5-1  shows  the  basic  relationship  of  T&E  in  the  system 
acquisition.  Developmental  testing  and  operational  testing  are 
run  largely  concurrently.  There  are  advantages  and  disadvantages 
to  this,  as  discussed  in  the  following  quotes  from  the  IDA/OSD 
Reliability  and  Maintainability  Study.  Volume  III,  Case  Study 
Analysis . 
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Source:  Navy,  Program  Manager's  Guide,  NAVMAT  P-9494,  Naval  Material  Command,  July  1983 

Exhibit  5-1.  RELATIONSHIP  OF  THE  T&E  PROGRAM  TO  THE  SYSTEM  LIFE  CYCLE 


"An  area  where  schedule  dependencies  is  very  important  is 
during  the  test  and  evaluation  phase.  In  the  case  of.  the  F-18, 
the  reliability  development  tests  were  run  concurrently  with  the 
total  aircraft  full-scale  development  tests.  The  same  generation 
of  hardware  was  tested  in  both  programs.  Some  additional 
problems  were  identified  in  the  development  test,  but  had  it  been 
run  earlier  in  the  program,  the  full-scale  testing  could  then 
have  been  done  on  the  next  generation  hardware,  thus  giving  feed¬ 
back  on  the  corrective  actions  found  and  implemented  during 
development  testing  and  identifying  problems  on  the  next  genera¬ 
tion  of  hardware  prior  to  delivery  to  the  fleet.  Corrective 
actions  found  during  the  concurrent  testing  were  not  incorporated 
until  later  deliveries.  Indications  from  these  later  deliveries 
show  the  APG-65  [radar]  to  have  a  field  reliability  approaching 
40  hours  versus  the  24  hours  reflected  in  the  case  studies.  Had 
the  development  test  been  run  earlier,  this  improvement  would 
have  shown  up  during  the  full-scale  development  test  and  on  the 
first  units  delivered,  but  the  fact  of  test  concurrency  allowed 
the  systems  to  be  fielded  earlier  than  with  sequential  testing." 

"If  the  dependent  work  is  begun  or  completed  prior  to  the 
independent  work  which  feeds  the  system,  the  amount  of  difficulty 
later  encountered  is  a  function  of  the  accuracy  of  the  assump¬ 
tions  made.  If  some  of  the  assumptions  are  significantly  in 
error,  management  is  faced  with  a  redesign  effort,  accepting  the 
consequences,  or  something  in  between.  Many  times  the  cost  and 
schedule  consequences  or  redesign  would  probably  be  unacceptable. 
Therefore,  something  is  done  which  produces  less  than  the 
desired  results.  Additionally,  the  work  done  out  of  sequence 
uses  the  available  resources  ineffectively.  An  example  of  this 
is  the  failure  modes,  effects,  criticality  analysis  (FMECA). 
When  this  is  done  off-line  after  PDR ,  it  is  unlikely  to  signifi¬ 
cantly  affect  design  for  diagnostics  or  the  elimination  of 
single-point  failures." 

While  the  situation  discussed  above  is  not  the  only  outcome 
of  FSD/production  concurrency,  it  indicates  a  set  of  circum¬ 
stances  which  can  occur.  There  are  two  studies  recently  com¬ 
pleted  by  the  Defense  Science  Board  regarding  transitioning  from 
development  to  production  which  address  implications  and  provide 
suggested  solutions  for  reducing  the  risks  of  these,  including 
applications  of  concurrency.  There  studies  are: 

•  Report  of  the  Defense  Science  Board  Task  Force  on 

Transition  of  Weapon  Systems  from  Development  to 

Production ,  May  1983;  and 
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•  -?Plvin<3  the  Risk  Equation  in  Transitioning  from  Devel¬ 
opment  to  Production,  Defense  Science  Board,  25  May  1983, 

Additional  sources  of  information  on  concurrency  and  T&E  activi¬ 
ties  are  referenced  in  Chapter  7  of  this  handbook. 

ILS  ELEMENT  DEVELOPMENT 

The  variety  and  distribution  of  ILS  elements  means  that  con¬ 
currency  among  them  is  practically  unavoidable.  In  applying 
concurrency  to  ILS  development,  the  major  concern  is  in  the  con¬ 
currency  occurring  between  the  hardware  and  software  design 
activities  and  the  ILS  system  development.  This  is  a  concern  due 
to  the  problems  with  data  availability  to  support  the  analysis. 

As  discussed  in  Chapter  2,  the  information  flows  in  a 
program  with  concurrency  are  significantly  more  disruptive  than 
m  a  program  with  very  constrained  concurrency.  This  means  that 
the  feedback  between  design  activities  is  often  limited.  Com¬ 
pounding  this  issue  is  the  difficulty  in  accessing  useful  data 
from  other  systems  or  from  the  test  results. 

In  the  preceeding  chapter,  the  connection  was  made  between 
the  system  design  activities  and  ILS  via  the  relationship  between 
R&M  and  the  diagnostic  system.  The  ability  to  develop  an  improved, 
mature  R&M  capability,  and  the  associated  system  diagnostics 
depends  on  being  able  to  incorporate  the  results  of  a  technol¬ 
ogy  maturation  program  in  the  architecture  and  development  of 
the  ILS/d iag nost i c  system.  Exhibit  5-2  shows  the  concurrencu  in 
the  development  and  production  schedules  of  the  F-16  radars. 
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Source :  IDA/QSD  Reliability  and  Maintainability  Study 


Exhibit  5-2. 


F-16  APG-66  PROGRAM  SUMMARY 


Exhibit  5-3  shows  the  overall  concurrency  in  the  F-16  C/D 
program.  Concurrency  has  been  used  throughout  this  program  in 
all  three  strategy  types:  Planning  or  Scheduling,  Management  and 
Acquisition.  The  logistics  concurrency  has  involved  the  concur¬ 
rent  development  of  the  critical  maintenance  capability  -  the 
Avionics  Intermediate  Support  (AIS)  facility,  and  the  non-AIS 
organizational  and  intermediate  support.  The  trade-offs  in 
developing  this  capability  have,  however,  led  to  the  need  for 
extensive  interim  contractor  support. 

Missile  systems,  on  the  other  hand,  can  tolerate  much  more 
concurrency  than  advanced  technology  aircraft  programs, 
primarily  because  of  the  limited  unknowns  in  the  technology.  The 
Peacekeeper  Program  has  been  able  to  sustain  extensive  concur¬ 
rency  because  it  has  been  able  to  build  on  the  base  of  experience 
from  the  Minuteman  Program.  This,  combined  with  an  extensive, 
well-run  management  information  feedback  system>  has  allowed  for 
the  significant  and  successful  use  of  concurrency. 

The  next  section  of  the  handbook  discusses  the  scheduling 
analysis  aspects  of  concurrency. 
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Part  III.  SCHEDULING  TECHNIQUES  AND  ANALYSIS 


Chapter  6.  Summary  of  Scheduling  Techniques 

Chapter  7.  Structure  of  Concurrency  Schedule  Analysis 

Chapter  8.  Analysis  of  Concurrency  Schedule  Risk 
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PART  III.  SCHEDULING  TECHNIQUES  AND  ANALYSIS 


This  part  of  the  handbook  focuses  on  the  major  scheduling 
techniques  at  the  disposal  of  Project  Managers  (Chapter  6). 

Also  included,  in  Chapter  7,  is  a  description  of  an  approach  for 
conducting  analysis  of  schedules  in  which  concurrency  is  to  be 
used.  Finally,  in  Chapter  8,  discussion  of  how  the  risks  asso¬ 
ciated  with  concurrency  can  be  analyzed  is  provided.  The  purpose 
of  these  chapters  is  to  provide  a  centralized  discussion  of  con¬ 
siderations  the  Program  Manager  should  be  aware  of  and  some 
approaches  for  conducting  analyses.  Appendix  E  contains  guidance 
on  developing  a  scheduling  network. 


m 
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CHAPTER  6.  SUMMARY  OF  SCHEDULING  TECHNIQUES 


•  Planning  and  Scheduling  as  Program  Functions 

•  Gantt  Charts 

•  Milestone  Schedules 

•  Networking 


CHAPTER  6.  SUMMARY  OF  SCHEDULING  TECHNIQUES 


PLANNING  AND  SCHEDULING  AS  PROGRAM  FUNCTIONS 


A  significant  concern  of  the  Program  Manager  is  the  develop¬ 
ment  of  a  program  plan  or  acquisition  strategy,  and  a  set  of 
program  schedules.  These  requirements  refer  to  related,  but 
separate  functions  within  the  program  office:  Planning  and 
Scheduling.  Collectively  these  functions  relate  to  a  set  of 
specific  tasks : 

•  defining  all  the  work  or  activities  to  be  accomplished 
by  all  program  personnel  funded  within  the  program 
budget; 

•  ordering  the  sequence  in  which  all  these  activities 
should  take  place; 

•  determining  the  material  and  personnel  resources  which 
will  be  required  to  accomplish  these  activities. 

•  utilizing  identified  resources  to  determine  the  time 
required  to  perform  these  particular  activities; 

•  •  summing  up  the  periods  of  time  identified  for  those 
activities  to  determine  chronologically  when  those 
activities  must  be  accomplished;  and 

•  being  prepared  to  rework  these  plans,  revising  the 
schedule,  redistributing  the  resources,  and  changing 

the  sequence  of  work  as  work  is  redefined  or  corrected. i/ 

Planning  focuses  on  the  identification  of  the  tasks,  activi¬ 
ties,  and  events  which  must  be  performed  in  order  to  complete  the 
program.  Scheduling  is  the: 


i/  Davis,  Hugh  E.  and  Young,  Robert  W. ,  Planning  and  Scheduling 
Enhancement  in  the  Acquisition  Process  at  the  Aeronautical 

Systems  Division,  Air  Force  Business  Research  Management  Center, 

Wright-Patterson  AFB ,  OH,  November  1982. 
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■u  •••Process  of  obtaining  the  information  on  how  long  a  job 
should  take,  relating  it  to  the  other  jobs  required  to  deliver 
the  product,  and  laying  out  [these]  data  in  a  specific  format  to 
show  when  it  must  be  done  to  fit  all  the  [known]  constraints . "1/ 

There  are  many  techniques  available  to  the  Program  Manager 
to  use  to  develop  program  schedules.  These  techniques  can  range 
in  degree  of  detail  provided,  amount  of  input  data,  number  of 
variables,  number  of  activities/events  which  can  be  monitored, 
and  variety  of  conditions  that  can  be  monitored.  The  decision¬ 
maker  must  determine  which  technique  is  most  appropriate  for  his 
program,  based  on  factors  such  as: 

•  the  size  of  the  program, 

•  the  amount  of  resources  available  for  performing  the 
scheduling  function,  and 

•  the  magnitude  of  activities  and  events  to  be  scheduled. 

Depending  on  these  variables,  the  Program  Manager,  or  more 

likely  the  Program  Control  Directorate  working  in  conjunction 
with  the  functional  directorate  and  the  laboratory/contractor , 
selects  a  scheduling  technique  or  techniques.  It  is  possible 
that  more  than  one  technique  may  be  used  in  the  same  program. 

The  master  schedule  may  be  maintained  using  one  graphic  techni¬ 
que,  while  functional  schedules  for  more  complex  areas,  such  as 
ILS,  may  use  a  more  detailed,  comprehensive,  tracking  approach. 
Generally  speaking,  however,  it  is  desirable  to  plan  to  use 
techniques  that  are  clearly  compatible  with  each  other  and  pro¬ 
vide  appropriate  levels  of  detail.  Too  much  detail  on  a  schedule 


2/  Devenney,  T.,  Mason,  W.  and  Snell,  W. ,  Program  Scheduling  Hand¬ 
book,  Business  Management  Division,  Electronic  Systems  Division 
(ESD/ACBB),  Hans com  AFB ,  MA,  March  1980. 
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can  be  just  as  useless,  or  confusing,  as  too  little  detail  if  it 
impedes  making  meaningful  decisions. 

In  this  chapter,  brief  summaries  are  given  of  the  major 
scheduling  techniques  available  to  the  program  office.  The  focus 
is  on  techniques  providing  differing  amounts  of  information  on. 
the  required  sequence  and  relationships  of  program  activities, 
and  their  status. 

GANTT  CHARTS 

Gantt  Charting  is  one  of  the  earliest-developed  approaches 
to  schedule  planning,  and  among  the  simplest.  Information  is 
portrayed  in  relation  to  the  horizontal  and  vertical  axes  of  a 
chart.  The  dependent  data  (activities,  events,  functions,  etc.) 
to  be  scheduled  are  listed  down  the  vertical  axis.  Time  units 
(days,  weeks  or  months)  are  listed  across  the  horizontal  axis.  A 
bar  is  used  to  indicate  the  planned  duration  of  the  activity, 
stretching  between  the  starting  and  ending  points  of  time. 

Exhibit  6-1  shows  several  variations  on  Gantt  Charts.  Exhibit 
6“la  shows  the  various  responsibilities  of  staff  members  during  a 
particular  period  of  time.  Exhibit  6-lb  shows  the  duration  of 
particular  activities  in  an  acquisition  program,  also  by  month, 
for  a  year.  Exhibit  6-lc  is  a  variation  on  this,  showing  activi- 
ties  for  increments  of  months  over  a  much  longer  period.  Unlike 
the  first  two  figures,  this  figure  also  illustrates  an  implied 
series  of  activities  that  are  sequentially  dependent  (i.e.,  the 
first  one  must.be  completed  before  the  second  one  can  be  begun.) 
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Exhibit  6-1.  EXAMPLES  OF  GANTT  CHARTS 
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This  type  of  schedule  is  frequently  called  a  "waterfall"  sched¬ 
ule.  Exhibit  6-ld  shows  the  Planning,  Programming  and  Budgeting 
System  (PPBS)  cycle  across  not  only  the  calendar  year  but  also 
across  the  fiscal  period.  This  is  a  frequently  used  approach  for 
illustrating  the  cyclic  nature  of  the  PPBS  since  the  same 
sequence  of  activities  is  repeated  annually  as  well  as  in  paral¬ 
lel  for  multiple  years. 

Exhibits  6-2  and  6-3  illustrate  two  actual  examples  of  the 
use  of  Gantt  Charts.  Exhibit  6-2  shows  the  suggested  period  of 
time,  in  the  form  of  acquisition  program  phases,  that  analytical 
techniques  should  be  used.  As  can  be  seen,  only  summary  level 
information  is  provided.  Additional  detail  would  have  to  come 
from  schedules  developed  for  each  of  the  template  activities. 

Only  general  temporal  relationships  are  shovn. 

In  Exhibit  6-3  more  information  is  provided  in  the  form  of 
codes  to  show  the  value  to  the  program  of  conducting  this  partic¬ 
ular  reliability-related  activity  in  a  given  phase.  As  with  the 
previous  example,  no  clear  relationship  between  the  activities  is 
illustrated,  although  the  start  and  end  points  allow  for  some 
interpretation.  In  this  example,  the  codes  have  been  used 
instead  of  a  uniform  bar  to  illustrate  when  an  activity  should  be 
performed  to  optimize  program  success. 

As  can  be  seen  from  these  two  examples,  the  amount  of  in'fior- 

/  .  ']' 

mation  that  can  be  conveyed  by  a  Gantt  Chart  is  fairly  limited. 

Although  more  information  concerning  status  can  be  conveyed  than 

has  been  shown,  usually  through  the  use  of  a  current  vertical 
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EXAMPLE  OF  GANTT  CHARTING: 
PATH  TEMPLATE  TIMELINES 
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Source:  Anderson,  R.T.,  Reliability  Design  Handbook,  ITT  Research  Institute, 

Chicago,  Illinois  ,  March  ""19  76 

Exhibit  6-3.  EXAMPLE  OF  GANTT  CHARTING: 

RELIABILITY  PROGRAM  ELEMENTS 
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time  line  with  annotating  comments,  the  overall  information  is 
limited. 

The  following  are  some  of  the  general  strengths  and  .weak¬ 
nesses  associated  with  Gantt  Charts. 

1.  Strengths 

"Perhaps  the  greatest  asset  of  a  Gantt  Chart  is  its 
simplicity  both  to  construct  and  to  communicate  information.  They 
are  easy  to  develop  and  update  and  the  mechanics  are  compatible 
with  pen  and  pencil  or  computer  controlled  automatic  plotters 
working  from  a  data  base.  The  sophistication  depends  on  the  pur¬ 
pose  and  the  size  of  the  job,  but  do  not  let  color  — coded  window 
dressing  influence  your  assessment  of  what  is  needed.  Grease  pen¬ 
cil  on  wall  charts  have  been  very  successful  and  the  fancy  pres¬ 
entations  require  many  hours  of  preparation. 

"The  Gantt  technique  is  quite  good  for  showing  the 
summary  levels  of  a  total  program  which  will  give  a  quick  one 
page  reference  for  the  phasing  of  all  major  program  activities. 
They  are  also  very  useful  for  expanding  a  single  schedule  task 
into  the  detailed  activities  that  make  it  up.  An  example  would 
be  to  show  the  time  phasing  of  effort  for  the  key  people  neces¬ 
sary  to  complete  an  activity. 

"The  Gantt  technique  works  well  if  the  activities 
scheduled  are  a  fairly  constant  level  of  effort  or  a  fixed 
resource  that  is  being  used  to  support  multiple  activities,  such 
as  test  facilities  or  equipment.  For  more  complex  activities  the 
Gantt  Chart  has  limited  capability. 

2.  Weaknesses 

"This  brings  us  to  what  the  Gantt  Charts  are  not  good 
for  and  the  greatest  weakness  is  the  inability  to  show  interac¬ 
tions  between  activities  except  on  a  very  simple  level.  Don't  try 
to  use  a  Gantt  display  to  lay  out  the  detailed  planning  for  any 
phase  of  a  program.  As  soon  as  the  number  of  activities  becomes 
large  (say  more  than  a  dozen)  the  Gantt  Chart  will  cause  more 
questions  to  be  asked  than  answered  about  the  relationship 
between  tasks. 

"They  also  fall  short  when  used  to  schedule  activities 
that  are  complicated  in  nature.  This  is  especially  true  when  the 
progress  display  feature  is  being  used.  In  other  words,  when  the 
activity  is  not  best  discribed  by  the  mere  passage  of  time  from 
start  to  finish,  then  a  Gantt  display  can  be  a  very  misleading 
way  to  monitor  performance.  The  following  are  a  few  examples  of 
the  confusion  that  can  arise. 
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The  task  is  front-loaded  in  terms  of  the  effort 
and  material  resources  required. 


75%  Complete 


time  now  50%  of  elapsed  time 


The  display  shows  an  ahead  of  schedule  situation 
when  it  is  in  fact  on-time;  it  should  be  75%  complete  at  this 
point.  In  some  cases  it  could  be  behind  schedule  depending  on  how 
front-loaded  the  task  is. 

b.  The  bulk  of  activity  is  required  toward  the  end  of 
a  task . 


30%  Complete 


time  now  50%  of  elapsed  time 


late . 


The  task  may  be  ahead  of  schedule,  but  shows  up  as 


"The  point  is  that  a  Gantt  technique  is  not 
designed  to  easily  show  completion  progress  on  complex  tasks  and 
you  can  spend  a  great  deal  of  time  explaining  artificial  per¬ 
formance  variances  which  detracts  from  the  analysis  of  the  real 
situation. 

"The  Gantt  approach  is  also  deficient  in  showing 
actual  performance  versus  the  original  planned  schedule  baseline. 
If  the  start  and  completion  dates  are  different  than  the  plan, 
there  is  no  simple  way  to  display  that  information."^/ 


MILESTONE  SCHEDULES 

Milestone  Schedules  are  a  variation  on  Gantt  Charts.  It 
expands  on  the  bar  chart  concept  by  including  milestones  or 
events  as  indicators  of  progress.  Schedules  are  developed  show¬ 
ing,  for  specific  functions  or  sets  of  activities,  key  events 

2/  This  discussion  of  strengths  and  weaknesses  is  extracted  from  the 
Program  Scheduling  Handbook,  ESD/ACBB  Hanscom  AFB,  MA,  March  1980. 
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and  when  they  are  intended  (or  must)  take  place.  Frequently  a 
set  of  milestones  for  a  specific  function,  such  as  program  plan¬ 
ning,  are  given  on  the  vertical  axis  with  symbols  used  to  indi¬ 
cate  the  planned  and  actual  status.  A  variation  on  this  is  to 
show  several  activities  on  the  time  bar.  The  milestone  schedule 
improves  on  the  Gantt  Chart  by  providing  information  on  the 
status  of- the  actual  performance,  showing  the  relationship  of  the 
actual  status  to  the  baseline  schedule,  and  allowing  for  changes 
in  the  future  plan. 

In  order  to  effectively  develop  a  milestone  chart,  signifi¬ 
cant  program  planning  must  be  accomplished.  Unlike  the  Gantt 
Chart,  which  focuses  on  very  summary-level  data,  the  milestone 
chart  is  intended  to  indicate  the  status  of  lower  level  activi¬ 
ties.  In  the  case  of  Gantt  Charts,  there  may  or  may  not  be  lower 
level  activities.  Milestone  charts  are,  in  the  context  of 
program  management,  intended  to  indicate  that  much  more  informa¬ 
tion  is  behind  them.  However,  like  the  Gantt  Chart,  the  mile¬ 
stone  schedule  is  a  status  monitoring  tool,  not  a  forecasting 
technique  (i.e.,  network  analysis).  One  cannot  extrapolate  the 
potential  status  of  activities  and  resources  given  the  informa- 

i 

tion  on  a  milestone  chart. 

A  major  use  of  milestone  schedules  is  to  construct  hier¬ 
archies  of  functional  schedules  for  programs.  They  can  be  used 
to  cross  reference  activities,  events,  functions,  staff  responsi¬ 
bilities,  etc.  and  can  be  very  easily  tied  to  the  Program  Work 
Breakdown  Structure  (WBS).  The  WBS  is  the  widely  used  structure 
for  segmenting  program  activities,  the  system,  and  subsystem 
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components,  into  specific  strata  and  functions.  Program  costs 
are  usually  related  to  WBS  elements,  as  are  many  risk  analysis 
techniques . 

Exhibits  6-4  through  6-6  show  examples  of  actual  applica¬ 
tions  of  milestone  schedules  from  the  F-16  Master  Program  Sched¬ 
ule.  Each  of  these  provides  varying  information  in  addition  to 

i  ! 

the  status  of  an  activity  for  a  specific  period  of  time.  Exhibit 
6-4  illustrates  the  status  of  several  F-16  Avionics  Intermediate 
Support  (AIS)  site  deliveries  as  of  a  specific  period  of  time. 

The  black  arrows  show  completed  deliveries,  while  the  white 
arrows  show  deliveries  still  to  be  made.  At  the  time  of  the 
analysis,  three  of  the  site  deliveries  were  behind  schedule.  In 
some  variations  on  this  type  of  chart  a  brief  explanatory  comment 
may  be  given  on  the  same  line,  providing  clarifying  information 
on  the  status. 

A  variation  on  this  is  to  note  interim  completion  values. 
These  values,  usually  in  the  form  of  percentages  of  work  com¬ 
pleted,  are  assigned  to  specific  interim  milestones  for  the 
activity.  These  milestone  values  are  assigned  when  the  schedule 
is  developed  and  may  be  used  as  the  baseline  with  which  to  com¬ 
pare  the  various  progress  calculations  as  the  schedule  advances. 

Exhibit  6-5  provides  somewhat  more  information.  For  each 
set  of  aircraft  to  be  delivered,  the  monthly  scheduled  and  actual 
deliveries  are  shown,  along  with  the  variation  (plus  or  minus) 
between  the  two.  The  total  projected  or  authorized  quantities 
are  also  shown  for  each  country.  The  monthly  status  is 
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Exhibit  6-5.  EXAMPLE  OF  GANTT  CHART:  F-16 
AIR  VEHICLE  PRODUCTION  SCHEDULE 


summarized  for  the  cummulative  schedule,  actual  and  variances 
given.  This  additional  information  allows  for  significantly 
9^®^ter  understanding  of  the  status  of  a  number  of  specific  and 
similar  activities  and  can  support  a  more  comprehensive  overview 
of  a  schedule.  However,  it  is  successful  largely  due  to  the 
similarity  among  the  activities. 

Exhibit  6-6,  also  from  the  F-16  Program  Master  Schedule, 
illustrates  still  another  variation  on  milestone  schedules.  In 
this  schedule,  bars  are  shown  in  multiple  patterns  to  illustrate 
different  tasks  within  a  given  category,  and  are  annotated.  Con- 
current  engine  testing  is  noted  for  the  F— 16B  No.  56,  with  alter¬ 
nating  test  and  modification  periods  shown  for  all  of  the  test 
aircraft.  Each  task  is  shown,  with  a  duration  and  in  some  cases 
contingency  conditions  noted  (e.g.,  F-16B  No.  82,  Avionics.) 
Compared  to  the  other  two  F-16  Master  Schedule  examples,  much 
more  specific  information  is  contained  on  this  schedule.  Such  a 
chart,  while  being  useful  for  illustrating  a  planned  schedule  is 
not  as  informative  on  the  status  of  progress  in  completing  the 
tasks.  That  information  would  most  probably  need  to  be  obtained 
from  schedules  for  the  next  lower  tier  of  activities. 

The  importance  of  these  examples  is  to  note  that  milestone 
schedules  can  be  used  to  effectively  communicate  specific  infor¬ 
mation  of  varying  types.  However,  it  is  also  easy  to  "overload" 
bar  charts  with  descriptive  information  so  as  t6  make  it  very 
difficult  to  quickly  determine  the  status  of  the  activities. 

The  following  brief  summary  of  the  strengths  and  weaknesses 
of  milestone  schedules  has  been  extracted  from  the  ESD/ACBB 
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Exhibit  6-6.  EXAMPLE  OF  GANTT  CHART:  F-16A/B  FLIGHT 

TEST  PLAN 


Program  Scheduling  Handbook.  While 


not  be  an  exhaustive 


* 


this  may 


accounting  of  these  characteristics,  it  does  clearly  and  con¬ 
cisely  describe  the  major  ones  of  concern  to  program  decision¬ 
makers.  Also  included  is  a  set  of  suggestions  on  cautions  to  the 
Program  Manager  concerning  developing  and  maintaining  schedules. 

1.  Strengths 

"As  with  the  Gantt  chart,  the  milestone  schedule  can  be 
a  very  effective  method  of  communication.  The  symboloqy  is  rela¬ 
tively  standard  and  simple  to  use.  It  also  allows  the  presenta- 
tion  of  actual  progress  against  a  baseline  plan  and  changes  in 
future  plans.  The  mechanics  to  construct  milestone  schedules  are 
also  relatively  simple,  although  it  may  seem  that  we  spend  too 
muc  of  our  time  with  stick-on  arrows  and  diamonds  and  the  AFSC 
form  103.  ..Most  of  our  contractors  use  milestone  schedules 
extensively  and  they  are  usually  the  type  submitted  for  the 
rogram  Schedule  contract  data  item  (data  item  description  (DID) 

DI-A  3007)  as  well  as  their  use  in  the  Cost/Schedule  Control 
by  stems . 


"As... more  complex  applications  of  milestone  schedules 
are  encountered  we  must  remember  that  this  scheduling  approach  is 
giving  us  a  limited  view  of  what  is  in  fact  a  network  of  activi¬ 
ties.  Milestone  charts  are  used  to  portray  results  derived  from 
network  analysis  or  a  line  of  balance  computation  which  is  a 
variation  of  networking. 

2.  Weaknesses 

"As  with  the  Gantt  chart,  a  major  weakness  of  the  mile¬ 
stone  technique  is  the  lack  of  a  mechanism  to  show  interdependen- 
C1uS^°f  interaction  between  activities.  Although  milestone 
schedules  are  used  extensively  on  complex  programs,  they  are 
usually  che  product  of  some  type  of  network  analysis.  In  fact, 
the  milestone  technique  is  a  good  way  to  display  various  areas  of 
a  network  in  a  more  familiar  form - The  danger  lies  in  our  ten¬ 
dency  to  focus  on  the  milestone  format  and  lose  sight  of  the  true 
complexity  of  the  relationship  between  the  tasks  we  are  dealina 
with  and  the  total  program.  y 


"...A  milestone  schedule  that  shows  all  of  the  major 
events  for  [a]  WBS  item...  is  backed  up  by  functional  schedules 
from  [each  of  the]  responsible  organizations.  Each  of  these 
organizations  must  accomplish  certain  key  activities  to  complete 
testing  for  this  system.  The  milestone  technique  will  not  tell 
us  what  the  sequence  of  activities  must  be,  where  the  constrain¬ 
ing  points  are,  and  which  activities  are  most  critical  for 
management  attention.  In  fact,  there  will  be  many  opinions  about 
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the  above  information,  but  the  manager  will  be 
out.  The  milestone  technique  will  not  show  it  to 


left  to  sort  it 
us . 


"This  type  of  problem  becomes  non-t,rivial  quickly  as 
the  WBS  level  being  scheduled  increases.  There  is  an  ever¬ 
present  danger  that  activities  which  show  up  on  product  (WBS) 
oriented  schedules  and  on  several  different  functional  schedules 
may  change  in  one  place  while  the  impact  in  other  areas  is  not 
realized  until  too  late.  Again,  milestones  do  not  depict  inter¬ 
actions,  the  manager  must  find  them  out. 


3.  Program  Manager  Cautions 


"Most  programs  require  the  contractor  to  submit  mile¬ 
stone  schedules.  The  contractor  is  generally  expected  to  select 
the  milestones  which  he  or  she  believes  will  indicate  the  overall 
status  of  activities.  The  data  item  description  normally  allows 
the  government  to  approve  of  the  milestones  which  are  selected 
for  periodic  reporting.  BEWARE  I  There  are  many  traps  in  this 
area.  First,  do  not  let  the  contractor  report  milestones  which 
no  one  in  the  SPO  understands.  Avoid  cryptic  abbreviations. 
Avoid  generating  a  host  of  alphabet  soup.  The  schedule  is  meant 
to  communicate  information.  It  can  not  do  that  without  careful 
selection  of  the  milestones.  If  you  don't  understand  every  line 
in  a  contractor's  schedule  have  it  explained  to  you. 


"Another  danger  to  avoid  is  the  tendency  for  "micro- 
management."  For  example,  one  of  the  writers  [of  the  Program 
Scheduling  Handbook]  had  a  contract  with  a  large  contractor  who 
had  automated  a  technique  for  producing  milestone  schedules. 
Working  with  the  SPO,  generic  milestones  were  identified  for 
reporting.  Since  the  SPO  software  engineers  had  expressed  a 
requirement  for  Computer  Program  Component  (CPC)  level  visibil¬ 
ity  below  the  Computer  Program  Configuratin  Item  (CPCI)  level,  a 
generic  set  of  eight  milestones  were  selected  for  each  CPC. 
Milestones  such  as  start  and  complete  design,  and  start  and 
complete  coding  were  at  first  considered  reasonable  types  of 
milestones.  Not  until  the  SPO  had  let  the  contractor  implement 
this  type  of  schedule  did  we  realize  that  there  were  more  than 
400  CPCs  and  that  the  software  milestones  would  therefore  number 
3200.  Similar  mistakes  were  made  in  other  disciplines  with  a  net 
result  of  a  program  milestone  schedule  which  had  10,000  events. 
Needless  to  say,  the  presence  of  so  many  "trees"  prevented  anyone 
from  even  finding  the  "forest."  Remember,  micro-manaqement  can 
kill. 


"Select  milestones  by  having  each  functional  Division 
Chief  determine  what  he  or  she  considers  important.  Keep  count 
of  the  milestones  that  he  is  requesting  and  avoid  the  CPC  report¬ 
ing  problem  above.  Get  feedback  on  the  utility  of  the  first  few 
schedules  from  each  division  and  revise  the  schedule  accordingly. 
Work  closely  with  the  contractor  in  developing  a  schedule  which 
will  communicate  information.  Ypur  requests  for  changes  to  the 
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contractor's  submission  can  be  made 
comments  on  the  data  item. 


as  part  of  your  official 


"Another  problem  with  milestone  schedules  is  in  deter¬ 
mining  whether  the  time  depicted  for  an  activity  is  reasonable. 
tuSte™XIle-r  iei?ce  on  other  programs  or  independent  assessments  by 


the  SPO 
because 
ence. 
Repor  t 
system, 
ness  of 


divisions  are  good  techniques,  but  tend  to  be  subjective 
of  the  lack  of  a  credible  data  base  of  historical  experi- 
H^o\er'  }f  y°Ur  contract  requires  a  Cost  Performance 
(LPR)  and  the  validation  of  the  contractor's  C/SCSC 
there  is  another  method  for  determining  the  reasonable- 
a  particular  schedule. 

,  .  "The  contractor's  C/SCSC  system  is  required  to  have  a 

scheduling  system  which  meets  certain  criteria.  An  essential 
criterion  is  the  ability  for  schedules  at  the  work  package  level 
o  support  and  track  to  the  schedules  at  the  cost  account  level, 
lmilarly,  the  cost  account  schedules  must  support  and  track  to 
the  intermediate  schedules  and  to  the  master  schedule.  For  any 
item  on  the  milestone  schedule,  the  contractor  should  be  able  to 
demonstrate  that  lower  level  schedules  support  this  item.  These 
lower  level  schedules  can  be  reviewed  upon  request.  If  you  are 
lucky  enough  to  have  a  data  accession  clause  on  your  contract, 
you  can  also  request  copies  of  the  lower  level  schedules  which 
support  a  particular  time.  However,  beware  once  again  of  micro- 
management.  A  $ 4 Om  contract  can  have  over  5,000  work  packaqes 
eac  with  a  schedule.  Insure  that  you  use  these  requests  spar¬ 
ingly  to  support  a  one-time  review  of  a  critical  area.  Do  not 
allow  yourself  to  get  dragged  into  the  trees.  On  the  other  hand, 
don  t  be  afraid  to  use  this  data.  Spot  checks  help  insure  that 
the  contractor's  schedules  are  credible. 


Most  milestone  schedules  from  the  contractor  contain  a 
narrative  describing  why  changes  occurred.  This  narrative  is 
never  complete  and  there  is  a  tendency  for  SPO  Divisions  to  bom¬ 
bard  Business  Management  with  data  item  comments  such  as  "Why  did 
this  slip"  or  "Why  will  this  item  take  three  months?"  Do  not 
accept  these  comments  blindly  and  forward  them  to  the  contractor. 
A  SPO  cannot  afford  to  communicate  with  the  contractor  solely  or 
even  primarily  through  the  monthly  schedule.  If  someone  has  a 
question,  let  him  call  his  counterpart  at  the  contractor's  and 
ask  the  question.  The  only  times  that  formal  questions  should  be 
included  in  your  data  comments  are  when  the  contractor  refuses 
to  provide  an  informal  answer  or  when  the  SPO  OPR  wants  the 
answer  formally  documented  for  some  reason.  These  questions 
should  be  limited.  Answering  these  questions  can  tie  up  a  signi¬ 
ficant  portion  of  the  contractor’s  management  resources  - 
resources  that  could  probably  be  better  applied  elsewhere. 

"Miiestone  schedules  require  a  significant  amount  of 
effort  to  generate.  Searching  out  the  status  of  an  item  consumes 
more  time  than  making  the  chart.  Answering  questions  adds  to  the 
time.  If  you  have  ever  developed  a  set  of  milestone  charts  for  a 
Program  Financial  Review  or  for  an  internal  review  you  can 


6-18 


* 


quickly  appreciate  the  amount  of  effort  required.  Remember  this 
when  you  ask  the  contractor  to  do  something. " 

NETWORK  ANALYSIS 

Network  analysis,  or  networking,  refers  to  the  group  of 
scheduling  techniques  which  are  designed  to  expand  on  the  basic 
activity/event/duration  scheme  of  Gantt  Charts  and  milestone 
schedules  by  including  information  on  event  relationships.  This 
is  somewhat  akin  to  adding  the  dimension  of  depth  to  a  previously 
two-dimensional  description.  Information  on  the  interdependen¬ 
cies  of  events  and  activities  and  the  precedence  relationships 
are  developed  and  described  in  scheduling  with  network  analysis. 

(As  noted  in  the  previous  discussion  of  milestone  schedule  weak¬ 
nesses,  networks  are  frequently  what  lay  behind  milestone  sched¬ 
ules.)  Exhibit  6-7  shows  this  evolution. 

There  are  a  wide  variety  of  network  analysis  techniques.  The 
most  well  known  and  widely  used  are  the  Critical  Path  Method 
( CPM )  and  the  Program  Evaluation  and  Review  Technique  (PERT). 

PERT  has,  in  turn,  generated  a  number  of  variations  including 
GERT  (Graphic  Evaluation  and  Review  Technique),  and  VERT  (Venture 

Evaluation  and  Review  Technique),  as  well  as  versions  emphasizing 

4  / 

different  evaluation  aspects  (i.e.,  TAC-PERT,  and  PERT-Cost ) . — ' 

There  are  certain  characteristics  that  most  network  analysis 

techniques  share,  and  it  is  useful  to  briefly  review  them  before 

i/  There  are  numerous  discussions  on  PERT,  its  variations  and 

applications.  Only  a  general  discussion  is  included  here,  with 
some  elaboration  on  risk  applications  of  PERT  in  Chapter  10.  For 
a  more  detailed  discussion  of  PERT,  one  of  many  possible  sources 
is  An  Analysis  of  PERT  in  Weapon  System  Acquisition,  by  Robert  F. 
Ewart  and  Donald  M.  Nanney  (AFIT,  September  1974). 
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GANTT  BAR  CHART 


b.  MILESTONE  CHART 


c .  PERT  NETWORK 


Source :  Ewart,  Robert  F.  and  Nanney,  Donald  M. ,  An  Analysis  of 

Program  Evaluation  and  Review  Technique  (PERT)  in  Weapon 

System  Acquisition,  AFIT,  September  1974 


Exhibit  6-7. 


DEVELOPMENT  OF  PERT 


NETWORK  FROM  GANTT  CHART 
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discussing  any  specific  techniques.  First,  network  analysis 
techniques  are  designed  to  integrate  subschedules  to  show  how 
portions  of  the  project  connect  to  one  another.  This  means  that 
dependency  relationships  among  activities/events  are  identified. 
Information  is  developed  on  activities  which  must  be  completed 
before  other  activities  can  be  initiated,  or  activities  which  are 
not  explicitly  dependent  on  other  activities.!  This  type  of 
information  is  pivitol  to  the  analysis. 

Second,  in  some  form  or  another,  networking  usually  involves 
some  form  of  graphic  representation  of  activities  and  events. 

This  can  take  the  form  of  nodes  and  arcs  or  lines  connecting  the 
nodes,  or  boxes  connected  by  directional  lines,  or  some  similar 
form.  In  the  first  case  the  node,  usually  a  circle  or  box, 
represents  an  event  or  milestone.  The  line  between  the  mile¬ 
stones  represents  the  activity  initiated  by  the  event  or  the  goal 
of  the  activity.  The  line  may  or  may  not  have  an  arrow  to  show 
dependency  of  the  activities.  The  line  frequently  has  the  name 
of  the  activity  and  the  planned  duration  in  days  or  weeks.  The 
events  are  identified  and  numbered,  with  the  latter  information 
to  show  the  sequence  or  order  of  the  events.  Variations  on  these 
may  also  include  resource  estimates,  mandays  or  manmonths,  or 
dollars.  Exhibit  6-8  is  an  example  of  a  simple  arc  and  node  network. 

The  other  major  form  for  graphically  depicting  the  network 
is  shown  in  Exhibit  6-9.  In  this  type  of  network  all  of  the 
information  is  provided  within  the  box,  and  the  lines  only  serve 
to  connect  the  boxes.  Dependency  relationships  are  indicated  by 
the  position  of  the  boxes  in  relation  to  one  another.  As  can  be 
seen  in  this  example  a  lot  of  different  information  can  be 
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Exhibit  6-8.  EXAMPLE  OF  ARC  AND  NODE  NETWORK 
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Source:  Computer  Supported  Network  Analysis  System  (CSNAS) ,  operated  by  AFALC/XRI 

Exhibit  6-9.  EXAMPLE  OF  GRAPHIC  NETWORK  ILLUSTRATION 


provided.  Each  box  represents  an  activity/event  group.  The 
dates  on  the  top  are  one  set  of  target  start  and  end  dates,  such 
as  f°r  earlY  start  and  completion  or  the  optimistic  plan.  The 
dates  on  the  bottom  of  the  box  are  for  a  less  optimistic  plan. 

The  information  inside  the  box  includes  the  name  of  the  activity/ 
event,  and  the  identification/sequence  number.  The  slashed 
number  in  the  lower  left  represents  the  task  duration  (weeks/ 
days),  while  the  percentage  of  the  task  completed  is  in  the  lower 
right.  Slashes  through  the  box  indicate  tasks  completed.  While 
not  all  automated  network  analysis  systems  may  depict  this  infor¬ 
mation  in  this  way-  it  is  desirable  to  collect  and  maintain  the 
data . 

A  third  characteristic  of  networks  is  that  they  allow  fore¬ 
casting  of  the  impact  of  different  conditions  on  the  total  sched¬ 
ule.  Changes  in  completion  date  or  resource  utilization  can  be 
incorporated  in  the  network  and  the  effects  of  these  changes  can 
be  projected.  Neither  Gantt  Charts  nor  milestone  schedules  can 
be  used  in  this  way,  largely  because  of  their  inability  to  depict 
dependency  relationships.  PERT  requires  estimates  of  low,  most 
likely  and  high  resource  requirements,  which  are  used  in  employ¬ 
ing  the  forecasting  aspect  of  the  technique. 

A  fourth  characteristic  of  networks  is  their  ability  to 
represent  the  activities/events  on  the  critical  path.  These  are 
the  tasks  which  determine  the  total  duration  of  the  schedule,  and 
in  which  there  can  be  no  slack  time  (differences  between  early 
and  late  start  and  completion  dates).  The  critical  path  is 
usually  the  focus  of  management  concern  because  schedule  slippage 
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of  tasks  on  the  critical  path  will  produce  a  similar  slippage  in 
the  final  milestone,  assuming  all  else  stays  the  same.  Network 
analysis  allows  for  the  identification  of  the  tasks  on  the 


critical  path.  Since  these  may  frequently  not  be  the  tasks  which 

would  have  been  considered  the  most  " important”  it  is  vital  to 

know  and  understand  their  impact  on  the  ultimate  goal.  It  is 

also  important  to  understand  that  as  the  effort  progresses  the 

composition  of  the  critical  path  can  change,  with  tasks  moving 

5/ 

off  and  on  the  path.  Guidance  on  the  development  of  a 
schedule  network  is  provided  in  Appendix  E. 

As  noted  earlier,  one  of  the  most  frequently  used  network 
analysis  techniques  is  PERT.  Since  its  original  introduction 
with  the  Navy's  POLARIS  program  in  1958,  PERT  has  fallen  in  and 


out  of  favor  as  a  scheduling  and  analysis  technique.  A  variation 
of  PERT/CPM  is  currently  available  for  ILS  planning  through 
AFALC/XRI .  This  system,  the  Computer  Supported  Network  Analysis 
System  (CSNAS),  is  currently  used  by  a  number  of  major  SPOs 
including  the  B-1B,  F-16  FMS ,  LANTIRN,  B-52  Deployment  Office, 
and  many  others .  PERT  is  defined  as : 

"A  deterministic  networking  technique  which  uses  a  time- 
oriented  critical  path  analysis  to  aid  in  the  planning  and  con¬ 
trolling  functions  of  management.  Deterministic,  in  this  sense, 
refers  to  the  fact  that  an  event  following  an  activity  cannot  be 
considered  accomplished  until  all  activities  leading  to  the  event 
are  completed,  and  all  activities  starting  at  an  event  will 
occur.  The  time  estimates  may  be  either  single  or  three  time 
estimates.  The  word  "PERT"  infers  PERT  TIME;  it  does  not  include 
PERT  COST.  [PERT  COST  is]  a  PERT  network  on  which  both  schedule 
and  costs  are  planned  and  controlled  on  a  common  framework .  "6./ 

— !  A  more  detailed  generic  discussion  of  network  analysis  is  con¬ 
tained  in  the  Program  Scheduling  Handbook,  ESD/ACBB.  Additional 
sources  are  also  listed  in  Appendix  A. 

— /  Ewart  and  Nanney,  An  Analysis  of  PERT  in  Weapon  System 
Acquisition . 
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The  major  distinction  between  PERT  and  CPM  is  the  more  sta¬ 
tistical  or lentatfon  of  PERT;  however,  generally  these  terms  are 
used  jointly  in  referring  to  the  major  network  analysis  techni¬ 
ques.  In  the  study  Project  Management  Through  S imulat ion , the 
pros  and  cons  of  using  deterministic  network  analysis  versus  more 
complex  simulation  techniques  which  incorporate  potential  uncer-' 
tainty  through  the  use  of  probability,  are  discussed.  While  many 
other  studies  are  available  which  consider  applications  of  these 
techniques,  this  study  considers  aspects  of  particular  pertinence 
in  light  of  the  goals  of  this  handbook.  The  following  discussion 
has  been  extracted  from  the  study. 


SHORTCOMING  OF  CRITICAL  PATH  TECHNIQUES 


"Although  experienced  project  managers  recognize  many  of  the 
project  uncertainties  they  face,  their  ability  to  cope  with 
tamty-related  problems  is  limited  because  of  the  shortcom- 
of  analytical  techniques.  The  basic  problem  is  that  conven¬ 
ts0!131  critical-path  techniques  such  as  PERT,  CPM,  and  other 
similar  techniques  which  are  now  receiving  widespread  use  for 
planning,  scheduling,  resource  analysis,  and  costing  —  are  all 
deterministic."  That  is,  they  depend  upon  single-value  data 
inputs  to  the  network  plan  and  analysis  and  therefore,  cannot 
account  for  the  many  uncertainties  which  are  inherent  to  real- 
life  project  performance.  The  inability  to  account  for  uncer- 

1*1/  makes  it  dlfflcult  to  identify  and  deal  with  the  complex 

Ctt°^  thatK  C3n  devel°P  among  the  project  activities. 
Unfortunately,  such  interactions  can  profoundly  affect  overall 
project  performance.  They  invariably  add  to  project  duration  and 
increase  its  cost  and  they  rarely  improve  project  performance. 

"The  magnitude  of  the  optimistic  bias  of  deterministic  net¬ 
work  analysis  results  is  determined  by  a  number  of  factors: 


The  particular  project  activity  network  configuration: 

a.  the  length  (time-duration)  of  the  longest  time 
path  relative  to  the  other  paths 


1/ 


fieler,  A.  M.,  Project  Management 
TRANSIM,  Department  of  Engineering 


Through  Simulation,  Project 
Systems,  UCLA,  December  1976. 
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b.  the  number  of  near-critical  paths,  i.e.  very  close 
in  time  duration  to  the  critical  path 

c.  the  number  of  common  nodes  among  the  paths  which 
have  some  level  of  criticality 

d.  the  number  of  activities  which  have  less  than  100 
percent  probability  of  occurring. 

2.  The  degree  of  variability  of  task  performance  of  activ¬ 
ities  on  the  critical  path  and  on  near-critical  paths. 

f 

3.  Allocation  of  scarce  resources  based  on  determinis¬ 
tically-determined  resource  requirements. 

"One  of  the  more  misleading  aspects  of  conventional  deter¬ 
ministic  methods  is  the  assumption  that  there  is  a  unique 
"critical"  path  (longest  time  path)  in  a  project  network.  Where 
individual  project  activity  performances  are  variable,  there  are 
usually  several  paths  which  could  prove  to  be  longest,  depending 
upon  the  eventual  realization  of  activity  times.  The  probability 
that  an  individual  activity  will  lie  on  the  longest  time  path  for 
a  specific  network  realization  is  a  measure  of  the  activity's 
"criticality."*  Accordingly,  the  activities  requiring  close 
managerial  attention  are  those  with  significant  criticality  -  it 
should  not  be  limited  to  those  on  the  single  critical  path  of 
deterministic  techniques.  In  a  typical  project,  as  many  as  50  to 
60  percent  of  the  project  activities  can  have  a  significant  level 
of  criticality." 

It  is  useful  at  this  point  to  recognize  that  PERT  and  the 
other  members  of  the  class  of  more  sophisticated  networking 
techniques  have  multiple  roles  in  program  management  and  control. 
They  not  only  fulfill  the  function  of  depicting  the  activities, 
events,  time  and  relationships  involved  in  accomplishing  the 
program.  They  also  can  be  used  as  more  dynamic  management  tools 
in  the  area  of  program  control.  In  this  respect  they  are  used  to 
develop  and  evaluate  alternatives  and  to  forecast  impacts  of 
changes  in  the  schedule  or  resource  allocation  plan,  and  to 


*"Activity  'criticality'  may  be  viewed  as  a  direct  measure  of  the 
activity's  sensitivity,  i.e.,  its  importance  to  overall  project 
performance.  Criticality  is  measured  in  percent  between  zero  and 
100;  the  higher  the  percent  criticality,  the  more  the  activity's 
impact  on  overall  project  performance." 
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evaluate  risk.  In  the  latter  capacity  particular  techniques  are 
discussed  in  Chapter  10. 

The  following  discussion  of  the  general  strengths  and  weak¬ 
nesses  of  network  analysis  techniques  has  been  adapted  from  the 
Program  Scheduling  Handbook  referenced  above. 

1.  Strengths 

.  "Network  schedules  give  us  the  logic  and  organizational 

structure  to  combine  many  activities  and  incorporate  the  complex¬ 
ly  the  relationships  between  tasks.  This  lets  us  gather 
schedule  information  from  all  available  source?  within  a  program 

s!!ow.  Kthat  data  in  3  graphical  form  that  clearly 

shows  what  we  know  about  the  way  the  program  will  be  executed.  At 

the  same  time  the  process  of  building  the  network  will  highlight 

w£fLWS  and  give  US  the  chance  to  fill  in  the  gaps 

where  possible.  y  H 

"A  network  is  ideal  for  the  planning  phases  of  a  pro- 
gram  when  there  is  still  some  flexibility  in  the  time-phasing of 

fo.rna  ;  90  network  analysis  of  a  program  sets  a  firm 

foundation  for  getting  things  moving  in  an  orderly  fashion  when 

•^e,?0°"ahead^S  reived  for  a  project.  If  this  initial  effort 
k!,  1  ln  Wlth  an  organizing  scheme ...  products  from  the  network 
fh  wnUce.Can  be,extracted  either  by  responsible  organization  or 
the  WBS  items.  We  can  thus  provide  sub-schedules  for  the  effort 
that  a  program  participant  must  accomplish  over  a  period  of  time 
(this  could  be  a  simple  milestone  chart  or  a  network  in  itself). 
These  products  are  well  suited  to  documents  such  as  the  Program 
Management  Plan  (PMP).  AFR  800-2  states  that  the  PMP  is  direc¬ 
tive  on  program  participants! 


"A  network  schedule  can  be  developed  at  any  level  of  detail 
the  more  detail  we  can  incorporate  the  greater  the  accuracy 


and 

of  the  schedule  predictions.  No  one  will  argue  much  with  a  very 
generlized  schedule  of  the  program,  but  as  soon  as  we  start  show¬ 
ing  the  level  of  detail  that  explicitly  defines  the  interactions 
between  groups  for  specific  tasks  there  will  be  plenty  of  feed- 
beck.  Some  of  the  exchanges  may  be  emotional  as  an  individual 
e®,  ,hls  area  of  responsibility  is  being  encroached,  but  they 
will  be  exchanges  of  information.  That  is  the  objective  in  the 
first  place  I 

^‘Network  analysis  can  be  used  at  any  phase  during  a  pro¬ 
ram  s  life  cycle,  but  the  objective  and  the  use  of  results  will 
ary  An  analysis  performed  during  the  Full  Scale  Engineerinq 
Development  phase  may  indicate  that  we  have  a  low  confidence  of 
meeting  the  directed  date  for  an  Initial  Operational  Capability 


6-28 


% 


(IOC).  That  may  be  a  very  unpopular  result,  since  most  program 
parameters  are  already  set  (especially  the  budgets).  But  this 
type  of  analysis  can  give  ideas  of  where  to  concentrate  on 
improving  schedule  performance.  It  may  also  show  us  that  current 
plans  cannot  be  accomplished  and  should  be  changed! 

2.  Weaknesses 

"There  have  been  strong  arguments  voiced  against  the 
use  of  large  network  schedules  as  a  management  control  technique. 
These  criticisms  have  centered  on  the  PERT  and  CPM  approaches, 
but  apply  to  networking  in  general.  Once  an  effort  is  underway  a 
large  network  can  become  unwieldy  to  maintain  as  plans  inevitably 
change.  PERT— type  efforts  to  control  entire  programs  or  con¬ 
tracts  within  a  program  can  collapse  of  their  own  weight.  The 
result  may  be  a  monthly  report  that  is  simply  a  minor  rehash  of 
the  initial  plan  and  not  descriptive  of  the  current  situation. 
With  networks,  bigger  is  rarely  better. 

"A  network  is  an  approach  for  setting  up  a  management  infor¬ 
mation  system.  There  are  other  ways  of  fulfilling  the  informa¬ 
tion  needs  of  project  management  (  i.e.,  planning,  organizing, 
directing,  and  controlling),  but  no  approach  will  work  unless  we 
properly  scope  the  amount  of  effort  needed  to  support  it  and  make 
a  conscious  decision  to  commit  those  resources.  A  ne twork -based 
system  can  require  a  relatively  large  amount  of  time  to  set  up 
and  maintain,  although  the  degree  of  the  application  can  be 
tailored  by  the  level  of  detail  incorporated  (and  that  is  [the] 
suggested  approach). 

"If  there  is  not  a  clear  commitment  by  upper  level  manage¬ 
ment  (SPO  Director  and  Division  Chiefs)  to  implement  and  use  a 
network  system  it  will  not  be  a  medium  of  information  exchange. 
The  management  style  of  an  organization  is  chosen  by  the  boss, 
either  explicitly  or  implicitly.  It  soon  becomes  obvious  to  the 
members  of  an  organization  what  informtion  the  boss  thinks  is 
important  and  this  is  where  the  real  effort  is  focused. 

"Computer  based  network  systems  are  the  next  area  of  prob¬ 
lems.  Large  network  applications  are  normally  automated  to  some 
extent  since  the  amount  of  data  being  maintained  is  extensive. 
Most  contractor  developed  schedules  are  some  form  of  automated 
network.  Although  there  are  efforts  in  process  to  provide  this 
type  of  computer  support  to  our  program  offices,  the  access  to 
these  resources  is  limited  at  this  point,  so  we  have  a  very 
limited  ability  to  handle  large-scale  networks  internally. 

"The  PERT  and  CPM  systems  have  also  been  criticized  for 
their  tendency  to  focus  attention  on  the  critical  path  only.... 
There  can  be  a  considerable  amount  of  "optimistic  bias"  in  the 
expected  time  predicted  by  the  single  critical  path  calculations. 
We  can  get  a  general  idea  of  the  extent  of  the  problem  for  any 
application  by  identifying  the  number  of  alternate  paths  through 
the  network  that  are  close  in  total  length  to  the  critical  path 
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(say  within  a  few  %).  The  slack  time  will  be  low  on  these  paths. 
If  there  are  multiple  paths  close  to  the  critical  path  length, 
then  the  expected  time  will  be  optimistic  to  some  extent  and  a 
more  complex  method  should  be  used. " 

As  can  be  seen  from  these  and  the  preceeding  evaluation  of 
network  analysis  techniques,  it  is  important  to  select  an  appro¬ 
priate  level  of  detail  with  which  to  represent  the  program  activ¬ 
ities,  and  to  take  precautions  against  embedding  overly  optimistic 
estimates  of  future  accomplishments. 
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CHAPTER  7.  STRUCTURE  OF  CONCURRENCY  SCHEDULE  ANALYSIS 


•  Program  Schedule  Components 

•  Overview  of  the  Analysis 

•  Description  of  the  Analytical  Process 
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CHAPTER  7.  STRUCTURE  OF  CONCURRENCY  SCHEDULE  ANALYSIS 


Given  that  a  Program  Manager  has  selected  a  scheduling 
technique  appropriate  for  the  character  of  his  program,  he  must 
also  be  able  to  analyze  the  impacts  of  applying  concurrency. 
Concurrency  can  be  used  in  a  number  of  different  ways  in  a 
P^°9rara/  as  discussed  in  Chapter  2.  Three  types  of  concurrency 
have  been  identified: 

•  normal  concurrency,  used  as  a  scheduling  strategy  for 
balancing  workload  and  personnel  assignments; 

•  PAanned  concurrency,  used  as  an  acquisition  strategy 
for allowing  an  early  IOC  or  for  reducing  risk;  and 

•  exceptional  concurrency,  used  as  a  management  strategy 
to  compensate  for ^unforeseen  problems  or  to  respond  to 
a  crisis. 

Regardless  of  the  type  of  circumstances  surrounding  the 
application  of  concurrency,  it  is  necessary  for  the  Program 
Manager,  or  more  likely  the  Program  Control  Directorate,  to  eval¬ 
uate  the  risks  of  concurrently  scheduling  certain  activities. 

This  will  usually  involve  the  Program  Control  Directorate,  or  the 
specific  functional  area,  developing  a  preliminary  schedule  of 
the  segment  of  the  program  involved  and  then  performing  a  series 
of  simulations  or  "what  if”  exercises  to  estimate  potential 
impacts . 

This  chapter  lays  out  a  structured  approach  for  analyzing 

/  1 

these  impacts  with  the  goal  of  producing  a  revised  schedule.'*"' 

i 

y  The  concurrency  analysis  structure  described  in  this  chapter  has 
been  adapted  from  the  following  report.  Imrley,  Patricia  A.,  et 
al»  Shortening  the  Acquisition  Cycle;  Research  on  Concurrency, 

TR-8124— 1,  Management  Consulting  &  Research,  Inc.,  Falls'  Church , 

Virginia,  30  September  1982. 
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PROGRAM  SCHEDULE 


COMPONENTS 


In  attempting  to  understand  what  concurrency  involves, 
specific  factors  and  criteria  must  be  developed  for  considering 
project  activities  and  decisions  required  of  the  Program  Manager. 
The  basic  components  in  creating  program  schedules  must  be  iden¬ 
tified.  The  program  activities  and  events  can  be  considered  in 
light  of  these  components . 

Specifically,  it  is  necessary  to  consider: 

^•V  acq'^is;'-^;’-on  phases  such  as  Concept  Explora- 
raent :  VaUdati°"-  Ful1  Develop- 

•  '  major  ^tegories  of  work  performed  in,  or 
Technic?  direction  of,  the  Program  Office  such  as 

HanigemeJt?  e?=frnt’  L°9iSti°S  M»"»9ement .  Business 

•  3™  Jre^S  ~  subtasks  of  functional  work  such  as  hard- 

are  design,  software  design,  test  and  evaluation, 
etc.,  under  Technical  or  Systems  Management; 

•  Events  -  end  points  such  as  document  delivery,  desiqn 

™Aes;z;;sflastones' and  initiati°n « 

•  Activities  -  efforts  involved  in  preparing  for  a  par¬ 
ticular  event,  or  following  a  starting  event,  such  as 

planfrand°n  °f  3  baseline  and  review  of  a  procurement 

•  Organizations  -  groups  responsible  for  performing  acti- 
contractor s . &S  ^  Pr°gram  °ffiCe  funCtional  groups  or 

These  must  be  presented  in  terms  of  time  in  order  to  produce  a 
schedule . 

In  addition  to  these  basic  components,  a  program  schedule' is 
individualized  based  on  two  other  considerations: 
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•  System  Type  -  generic  type  of  weapon  system  related  to 
the  program  schedule,  e.g.,  aircraft,  missile. 

•  Subsystems  —  level  two  work  breakdown  structure  ele¬ 
ments  of  hardware,  as  defined  by  MIL-STD  881 A,  which 
may  be  on  different  developmental  schedules,  but  which 
collectively  constitute  a  viable  weapon  system. 

Exhibit  7—1  illustrates  an  example  of  a  work  breakdown  structure. 

Schedules  must  be  developed  for  work  in  each  of  the  three  levels, 

with  the  Level  1  schedule  corresponding  to  the  master  program 

schedule.  Level  3  schedules  are  aggregated  in  an  integrated 

summary  schedule  for  the  Level  2  subsystem. 

In  examining  the  degree  of  desirable  concurrency  for  a 
particular  program  many  factors  must  be  considered.  The 
following  considerations  are  briefly  summarized  here: 

•  factors  influencing  the  applicability  of  concurrency, 

•  acquisition  cycle-related  probl^rus, 

•  previously  suggested  alternatives, 

•  pros  and  cons  of  increased  concurrency,  and 

•  factors  for  changing  program  concurrency. 

It  is  not  clear  that  concurrency  is  applicable  to  all  system 
acquisitions.  Development  factors  such  as  design  status,  famili¬ 
arity  of  technology,  environmental  characteristics,  program 
office  personnel  experience,  and  contractor  availability/experi¬ 
ence?  and  production  factors  such  as  production  resource  availa- 
bility/manuf acturing  capability,  and  level  of  previous  program 
involvement  are  all  important.  But  so,  too,  is  the  discipline 
required  (risk  management)  of  scheduling  far  in  advance  of  an 
actual  requirement  (i.e.,  consider  production  and  logistics 
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Source:  Navy  Program  Manager's  Guide,  P-9494,  Naval  Material  Command,  July  1983 


Exhibit  7-1.  THREE-LEVEL  WORK  BREAKDOWN  STRUCTURE 


# 


problems  very  early  in  the  cycle).  Risks  of  technological 
advancement  or  lack  of  maturity  of  design  balanced  against  high 
development  cost  or  high  cost  uncertainty  can  doom  a  program  and 
require  higher  costs  of  maintaining  low-risk  alternatives.  There 
is  a  complex  hierarchy  of  resppnsibi lity  and  review  that  also 
contributes  to  the  problem  rather  than  to  the  solution.  In  addi¬ 
tion  to  these  needs  the  program  schedule  must  also  be  analyzed  in 
terms  of  its  sensitivity  to  external  forces  such  as  political  and 
budgetary  decisions. 


OVERVIEW  OF  THE  ANALYSIS 

The  approach  taken  in  developing  an  analytical  structure  for 
the  PM  to  use  in  making  concurrency-related  decisions  has  been  to 
construct  a  logical  framework  for  utilizing  a  progressive  accumu¬ 
lation  and  refinement  of  data.  The  analysis  is  structured  to  be 
neither  weapon  system  specific,  nor  sensitive  to  a  particular 
level  of  detail.  Rather,  it  is  applicable  to  any  system,  with 
appropriate  tailoring,  and  any  level  of  organizational  detail. 

Exhibit  7-2  shows  the  structure  of  this  analysis.  This 
structure  is  composed  of  seven  basic  steps  to  be  performed  by,  or 
under  the  direction  of,  the  Program  Manager.  The  first  step 
involves  the  development  of  the  initial  program  schedule  which 
forms  the  basis  for  concurrency  and  cost/schedule/readiness  risk 
analysis.  It  also  includes  the  formulation  of  the  rules  and 

for  performing  the  analyses,  and  the  identification  of 
an  initial  set  of  concurrency  options. 
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Evaluate  Funding  & 
Schedule  Constraints 


Determine  Motivation  for 
Concurrency:  Schedule  Pro¬ 
tection  or  Schedule  Compress- 
_ _ ion _ 


Determine  Magnitude  of 
Acceptable  Cost  Risk/ 
Schedule  Risk 


Develop 

Alternative  Schedules 


Evaluate  Risk 
For  Each  Alternative 


Select 

New  Schedule 


Exhibit  7-2.  CONCURRENCY  ANALYSIS  STRUCTURE 
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Having  set  up  the  problem,  the  second  step  concerns  the 
considerations  of  the  constraints  to  which  the  PM  must  respond  in 
the  schedule.  These  constraints  may  be  pre-existing  or  newly 
imposed,  endogenous  or  exogenous  to  the  program.  This  step  is 
closely  related  to  the  third  step,  determining  the  reason  for 
considering  concurrency.  In  evaluating  the  constraints  the  PM 
must  determine  the  desirable  scope  of  the  concurrency,  i.e.,  the 
phases,  functions,  task  areas,  and  activities  affected  by  the 
implementation  of  concurrency.  In  recognizing  the  motivation, 
the  PM  is  also  considering  the  ultimate  purpose  to  be  achieved  by 
using  concurrency  as  a  scheduling  mechanism,  as  well  as  the  cir¬ 
cumstances  driving  the  decision,  i.e.,  earlier  schedule  slippage, 
protection  of  the  remaining  schedule,  incorporation  of  changing 
direction,  etc. 

In  the  fourth  step,  the  PM  determines  the  magnitude  of 
acceptable  risk  to  be  considered  in  developing  and  selecting 
alternatives.  This  narrows  down  the  set  of  possible  alternative 
schedules  which  could  fulfill  the  requirements.  It  is  at  this 
point  that  decisions  are  made  about  acceptable  degrees  of  con¬ 
currency  and  risk.  Based  on  the  analysis  performed  in  the 

steps,  it  is  possible  that  there  may  be  more  than  one 
set  of  concurrent  activities  in  an  alternative,  each  of  which 
will  have  to  be  decided  upon. 

The  fifth  step  involves  the  development  of  alternative 
schedules  which  are  within  the  scope  of  the  preceeding  con¬ 
straints  and  risks.  A  variety  of  alternatives,  addressing  one  or 
more  of  the  previously  selected  sets  of  concurrency  options,  may 
be  developed. 


7-7 


'll 


The  companion  to  this  step  is  the  analysis  of  the  risks 
associated  with  each  alternative,  performed  in  the  sixth  step. 

The  evaluation  of  the  alternatives  is  performed  using  checklists 
tailored  to  the  particular  characteristics  of  the  system  type, 
the  stage  in  the  development  of  the  system,  and  the  particular 
task  areas  and  activities  involved.  Development  of  these  struc¬ 
tured  checklists  is  begun  with  the  selection  of  the  concurrency 
options  in  step  one  and  is  continued  through  each  step,  incorpor¬ 
ating  the  refined  direction  that  is  being  developed  in  this 
process.  They  are  tailored  to  respond  to  the  information  the  PM 
needs  to  make  an  actual  decision. 

Having  evaluated  and  scored  the  alternative  scheduling 
options,  the  final  step  is  the  selection  of  the  alternative  which 
most  adequately  satisfies  the  requirements  at  the  time  of  the 
decision.  Using  the  basic  criteria  developed  in  the  first  step, 
and  refined  for  the  actual  decision,  the  PM  trades-off  the 
options  presented  in  the  alternatives  among  cost,  schedule, 
readiness,  and  risk  in  the  program  environment.  The  ultimate 
selection  is  the  revised  schedule.  Although  a  single  alternative 
may  be  selected  in  this  process,  it  is  often  the  case  that  other 
viable  alternatives  have  been  developed  and  should  be  monitored 
in  the  process  of  subsequent  schedule  reviews. 

Initially,  several  assumptions  are  made: 

•  the  Program  Manager  is  assumed  to  have  a  Baseline 
Schedule ; 

•  funding  and  schedule  constraints  can  be  defined; 

•  resource  estimates  (e.g.,  of  time,  cost  and  manpower 
levels)  can  be  made  for  each  schedule  component; 
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•  analysis  of  alternative  schedules  representing  rela¬ 
tively  fixed  performance  will  be  performed;  and 

•  concurrency  can  be  meaningfully  considered  in  terms  of 
potential  savings  in  time  versus  risk. 

The  Top  Level  Hypotheses  (TLH )  are  simply  that: 

•  program  schedules  can  be  quantitatively  and  qualita¬ 
tively  evaluated; 

•  quantitative  or  qualitative  risk  analysis  measures  can 
be  developed  and  applied  to  evaluate  degrees  of  project 
concurrency;  and 

•  the  Program  Manager  can  himself  make  meaningful  deci¬ 
sions  regarding  shortening  the  program  acquisition 
cycle  using  a  structured  checklist  methodology. 

Given  these  hypotheses,  the  PM  must  be  able  to  intelligently 

apply  available  analytical  techniques  to  his  program  in  order  to 

make  concurrency  decisions. 


DESCRIPTION  OP  THE  ANALYTICAL  PROCESS 

The  following  is  a  brief  description  of  the  analytical 
process  represented  by  the  steps  listed  in  Exhibit  7-3. 

Step  1;  Construct  Baseline  Schedule 

Purpose:  Construct  foundation  for  making  decisions  on 

program  schedules  by  performing  initial  analysis. 


Approach: 

1.1  Develop  program  schedule  philosophy  (PSP) 

1.2  Construct  baseline  network 

1.3  Identify  potential  concurrency  options 

1.4  Develop  structure  of  risk  evaluation  checklists 

The  first  step  in  addressing  the  problem  of  concurrency  is 
to  identify  the  specific  characteristics  of  the  program  which 
must  be  identified  in  order  to  make  decisions  on  adjustments  to  the 
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Step  1. 


Step  2 


Step  3 


Step  4. 


Step  5. 


Step  6. 


Step  7. 


Construct  Baseline  Schedule 

1.1  Develop  Program  Schedule  Philosophy 

1*2  Construct  Baseline  Network 

1.3  Identify  Potential  Concurrency  Options 

1.4  Develop  Structure  of  Risk  Evaluation  Checklists 

Evaluate  Funding  and  Schedule  Constraints 

2.1  Determine  Significance  of  Constraints 

2.2  Determine  Scope  of  Concurrency 

2.3  Relate  Constraints  to  Concurrency  Options 

Determine  Motivation  of  Concurrency:  Schedule 
Protection  or  Schedule  Compression 

3  '  i  R E?tent  of  Internal  Program  Limitations 

3.2  Refine  Baseline  Schedule  Estimates 

3.3  Reevaluate  Preceeding  Decisions 

3.4  Develop  Initial  Set  of  Risk  Evaluation  Checklists 

Determine  Magnitude  of  Acceptable  Cost  Risk/Schedule  Risk 

4*7  e£eloP  Flaal  Baseline  Resource  and  Schedule  Estimates 

4.2  Determine  Acceptable  Degree  of  Concurrency 

4.3  Determine  Acceptable  Degree  of  Risk 

4.4  Review  Remaining  Concurrency  Options 

Develop  Alternative  Schedules 

5.1  Select  Constrained  Concurrency  Options  to  be  Used 
in  Developing  Alternatives 
Group  Concurrency  Options  for  Development  of 
Alternatives 

Generate  Alternative  Schedules 
Determine  Critical  Path  for  Each  Alternative 


5.2 

5.3 

5.4 


6.2 

6.3 

6.4 


Evaluate  Risk  for  Each  Alternative 

6.1  Finalize  Evaluation  Checklists 
Appiy  Checklists  to  Detailed  Schedule  and  Subschedules 
Score  Bach  Alternative  Based  on  Cost  and  Schedule 

Risk  and  Response  to  Constraints 
Aggregate  Data  to  Decision-Making  Level  of  Detail 

Select  New  Schedule 

7.1  Review  and  Revise  Decision-Making  Criteria 
Review  and  Revise  Proposed  Schedule-Monitoring  Techniaues 
Analyze  Results  of  Risk  Analysis  of  Alternatives 
Apply  Decision-Making  Criteria  to  Viable  Alternatives 
Select  Alternative 


,2 

i  3 

4 

5 

6 


Revise  Existing  Schedule 


Exhibit  7  3.  STEPS  IN  CONCURRENCY  ANALYSIS 
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schedule.  Developing  the  program  schedule  philosophy  involves 
construction  of  the  policy  and  procedures  or  rules  for  organizing 
the  analysis,  and  construction  of  the  criteria  for  making  sched¬ 
uling  adjustment  decisions.  It  also  includes  determination  of 
the  level  of  specificity  of  the  on— going  analysis,  a  reevaluation 
schedule  for  the  program  schedule  throughout  the  acquisition,  and 
a  description  of  basic  information  requirements  necessary  for  the 
analysis.  This  philosophy  is  subject  to  refinement,  as  is  the 
schedule . 

After  developing  the  basic  rules  for  considering  the  program 
schedule,  the  next  substep  is  the  actual  identification  of  the 
vi-i-i. es  and  events  to  be  scheduled,  the  development  of 
projected  values  for  each?  the  identification  of  the  tasks  and 
subtasks  which  compose  each  activity;  and,  finally,  the  arranging 
i-liis  information  in  a  set  of  networks,  reflecting  various 
levels  of  detail. 

The  third  substep  in  constructing  the  schedule  foundation  is 
the  identification  of  concurrency  options.  Not  all  of  the  acti- 
and  events  in  a  program  schedule  can  be  concurrently 
scheduled.  Therefore,  it  is  vital  to  identify  for  each  schedule 
(the  initial  as  well  as  subsequent  revisions)  those  activities 
and  events  which  cannot  be  reordered  or  adjusted.  Although 

identified  when  the  program  schedule  is  constructed, 
the  concurrency  options  must  be  reevaluated  as  portions  of  the 
schedule  are  completed. 

The  final  substep  in  the  initial  organization  of  the 
analysis  is  the  development  of  the  basic . structure  of  the  risk 
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evaluation  checklists.  These  checklists  will  be  used  in  Step  6 
to  evaluate  the  alternatives. 

— -S  -2i- — Evaluate  Funding  and  Schedule  Constraints 

Purpose:  To  determine  the  potential  scope  of  the  concur¬ 

rency  requirements,  based  on  specific  funding, 
readiness,  and  schedule  constraints. 


Approach: 

2.1  Determine  significance  of  constraints 

2.2  Determine  scope  of  concurrency 

2.3  Relate  constraints  to  concurrency  options 

In  this  step  the  actual  analysis  of  concurrency  potentials 
is  begun.  The  first  step  primarily  concerns  the  development  and 
organization  of  information  in  a  manner  useful  to  further  analy¬ 
sis.  This  second  step,  evaluation  of  constraints,  involves  the 
further  refinement  of  direction  through  a  three-step  process. 

A  basic  assumption  underlying  this  analytical  process  is 
that  schedules  should  and  must  be  re-evaluated  because  they 
incorporate  an  approach  which  may  no  longer  be  appropriate. 
Schedule  inappropriateness  may  be  due  to  a  variety  of  reasons 
(more  specifically  considered  in  Step  3).  However,  it  can  be’ 
translated  into  constraints  which  reflect  changes  in  resource 
requirements  or  demands.  These  constraints  may  be  due  to  circum¬ 
stances  within  the  program  or  outside  it.  They  may  take  the  -form 
of  restrictions  on: 

•  the  amount  of  time  remaining  to  accomplish  an  activity 
in  any  of  the  schedule  levels, 

•  the  projected  cost  allowed  to  complete  development, 

•  availability  of  organizations  to  perform  the  work,  or 

•  the  projected  level  of  risk. 
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The  characteristics  of  the  constraints  will,  in  turn, 
influence  the  potential  scope  of  the  concurrency.  The  scope 
relates  to  how  extensive  the  concurrency  may  be,  spannning 
phases,  functions,  task  areas,  activities  or  organizations.  Less 
significant  constraints  may  allow  for  restricting  the  scope  of 
the  concurrency  to  a  few  activities  at  the  sub-schedule  level. 

The  more  significant  the  constraints,  in  terms  of  total  program 
resources,  the  more  extensive  the  scope  of  the  concurrency.  The 
scope  is  tentatively  determined  in  this  sub-step  and  refined,  if 
necessary,  as  the  analysis  progresses. 

The  final  substep  in  Step  2  involves  relating  the  con¬ 
straints  to  the  concurrency  options  (identified  in  the  first 
step)  within  the  tentative  scope  of  the  concurrency  determined 
above.  Many  of  the  original  concurrency  options  previously 
identified  will  be  eliminated,  since  they  are  outside  the  scope 
of  the  projected  concurrency  requirements. 

Step  3:  Determine  Motivation  for  Concurrency:  Schedule 

Protection  or  Schedule  Compression 

Purpose:  Determine  the  amount  of  flexibility  and  limita¬ 

tions  existing  within  the  program  relating  to 
alternatives  open  to  the  PM. 

Approach: 

3.1  Determine  extent  of  internal  program  limitations 

3.2  Refine  schedule  uncertainty  and  dependency  estimates 

3.3  Reevaluate  previous  decisions 

3.4  Develop  initial  set  of  risk  evaluation  checklists 

This  step  is  iteratively  related  to  the  preceeding  step.  In 
this  step  peculiar  characteristics  and  conditions  within  the 
program  are  considered.  Particular  consideration  is  given  to  how 


e  potential  options  for 


they  may  influence  or  further  constrain  th 
developing  alternative  schedules.  There  are  four  substeps  in 
this  part  of  the  analysis.  The  first  three  substeps  are 
performed  and,  if  necessary  as  a  result  of  these  analyses,  the 
decisions  made  in  Steps  1  and  2  are  revised  to  take  into  account 
these  additional  constraints. 

The.,  first  substep  is  directed  toward  identifying  specific 
constraints  which  are  known  to  exist  within  the  program.  Some  of 
these  constraints  may  prohibit  rescheduling  or  reordering  of 
activities  and  events  which  would  otherwise  be  viable  concurrency 
options.  There  are  a  variety  of  conditions  which  could  produce 
this  effect  including  already  slipped  schedules,  previously 
completed  activities,  or  activities  already  in  progress  which 
cannot  be  redirected  or  rescheduled.  This  analysis  will  reveal 
the  general  orientation  of  the  planning  toward  schedule 
protection  or  schedule  compression • 

In  the  second  substep  the  preliminary  estimates  on  the 
degree  of  uncertainty  and  the  dependency  of  activities  and  events 
are  reevaluated  and  refined,  if  necessary,  to  reflect  the  addi¬ 
tional  understanding  of  the  program  constraints.  Related  to  this 
is  the  third  substep  in  which  previously  made  decisions  on  con¬ 
currency  options  and  the  checklist  structure  are  reevaluated  and 
modified,  if  necessary.  Finally,  an  initial  set  of  checklists  is 
developed  as  a  result  of  this  analysis.  These  checklists  are 
tailored  to  address  the  cost  risk  and  schedule  risk  associated 
with  the  options  used  to  generate  the  alternatives. 
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Step  4:  Determine  Magnitude  of  Acceptable  Cost  Risk/ 
Schedule  Risk  ~  “ 

Purpose:  Finalize  draft  decision-making  criteria  and  para¬ 

meters  for  selecting  alternate  schedules. 

Approach : 

4.1  Develop  final  baseline  resource  and  schedule  estimates 

4.2  Determine  acceptable  degree  of  concurrency 

4.3  Determine  acceptable  degree  of  risk 

4.4  Review  remaining  concurrency  options 

In  this  step  the  basis  for  developing  the  schedule  alterna¬ 
tives  are  further  refined  and  additional  detail  is  developed.  In 
the  first  substep  the  estimated  resource  requirements  for  accom¬ 
plishing  the  remainder  of  the  program  schedule  are  reviewed  and 
final  modifications  are  made.  These  estimates  are  for  the  cost, 
time  and  manloading  required  for  each  activity  and  event  in  the 
detailed  schedule. 

Based  on  these  estimates,  the  degree  of  concurrency  deemed 
to  be  acceptable  is  determined  in  the  second  substep.  The  degree 
of  concurrency  is  based  on  the  amount  of  overlap  a  dependent  or 
successor  activity  has  with  its  predecessor  activities.  The 
degree  of  concurrency  acceptable  to  the  PM  will  influence  the 
amount  of  risk  associated  with  a  particular  concurrency  option. 

In  determining  the  acceptable  degree  of  concurrency  the  PM  can 
decide  an  overall  amount  for  the  program,  such  as  “no  more  than 
10% ",  as  well  as  acceptable  amounts  for  each  concurrency  option, 

I 

based  on  the  perceived  risks  associated  with  each. 

In  addition  to  determining  the  acceptable  degree  of  concur¬ 
rency  or  amount  of  overlap  among  activities,  it  is  also  necessary 
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to  determine  the  limits  of  the  risks  the  PM  is  willing  to  toler¬ 
ate  in  shortening  the  acquisition  cycle.  Of  particular  interest 
are  cost  risk,  readiness  risk  and  schedule  risk,  and  the  rela¬ 
tionship  between  the  three.  In  this  substep  the  PM  makes  a 
preliminary  determination  of  the  limits  of  risk  and  the  circum¬ 
stances  under  which  additional  risk  will  be  undertaken. 

The  last  substep  is  the  final  review  of  the  remaining  con¬ 
currency  options.  Given  the  preceeding  analysis,  it  is  possible 
that  some  of  the  initial  concurrency  options  may  be  eliminated  or 
further  constrained.  It  is  important  to  determine  that  before 
proceeding  further  in  the  development  of  alternative  schedules, 
since  those  constrained  options  form  the  basis  for  constructing 
the  alternatives. 

Step  5:  Develop  Alternative  Schedules 

Purpose:  Translate  sets  of  concurrency  options  into  actual 

scheduling  alternatives  capable  of  being  evaluated 
in  terms  of  cost  and  schedule  risk. 

Approach : 

5.1  Identify  constrained  concurrency  options  to  be  used  in 
developing  alternatives 

5.2  Group  concurrency  options  for  development  of  alterna¬ 
tives 

5.3  Generate  Alternative  Schedules 

5.4  Determine  Critical  Path  for  each  alternative 

In  this  step  the  actual  alternative  schedule  or  revisions  to 
the  baseline  schedule  are  developed  and  prepared  for  further 
analysis.  In  order  to  do  this  the  first  substep  involves  deter¬ 
mining  which  of  the  remaining  concurrency  options  will  be  used 
as  the  basis  for  generating  alternatives.  It  is  conceivable  that 
not  all  of  the  options  will  be  applicable  and  an  effort  should  be 
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made  to  identify  those  that  are  not*  The  potentially  large 
resources  required  to  generate  alternate  schedules  make  that 
identification  worthwhile. 

t 

Having  identified  which  options  will  be  used,  the  next  sub¬ 
step  involves  arraying  the  options  in  alternative  groupings.  It 
is  possible  to  generate  a  variety  of  alternatives  by  varying  the 
combination  of  concurrency  options  and  the  projected  values  and 
schedule  for  each.  It  is  at  this  point  that  the  PM  has  the 
greatest  opportunity  to  be  innovative,  examining  the  specific 
needs  of  each  option  and  determining  the  minimum  requirements  to 
begin  each  activity.  These  innovative  approaches  are  considered 
in  the  context  of  the  acceptable  amount  or  degree  of  concurrency 
and  risk  determined  in  Step  4. 

Having  developed  the  base  for  each  alternative,  the  actual 
alternative  schedules  can  now  be  generated.  As  part  of  this 
process  it  is  worthwhile  to  review  the  preceeding  analysis  to 
insure  that  all  of  the  internal  and  external  constraints,  as  well 
as  previously  developed  direction,  are  accounted  for  in  the 
alternative  schedules. 

The  final  substep  in  this  portion  of  the  process  is  the 
determination  of  the  critical  path  in  each  of  the  alternatives. 

It  is  possible  at  this  point  that  some  of  the  alternatives  could 
be  eliminated  from  further  consideration  due  to  the  construction 
of  the  critical  path. 

Step  6;  Evalute  Risk  for  Each  Alternative 

Purpose:  Analyze  alternative  schedules  as  approaches  for 

responding  to  constraints. 
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Approach : 


6.1  Finalize  risk  evaluation  checklists 

6.2  Apply  checklists  to  detailed  schedules 

6.3  Score  each  alternative  based  on  cost  and  schedule  risk 

and  response  to  constraints  9 

6.4  Aggregate  data  to  decision-making  level  of  detail 

In  this  step  the  alternative  schedules  generated  in  Step  5 
are  evaluated  to  determine  their  appropriateness  as  approaches  to 
dealing  with  the  new  constraints.  The  major  mechanism  for  doing 
this  is  a  set  of  evaluation  checklists,  tailored  to  particular 
phases,  functions,  task  areas  and  activities  of  interest  in  the 
particular  analysis.  The  first  sub-step  in  this  evaluation  is 
finalizing  the  checklists  initially  developed  in  Step  3.  The 
final  version  of  the  checklists  should  be  tailored  to  address  the 
particular  activities  and  events  which  have  been  manipulated  in 
the  alternative  schedule.  They  must  be  designed  to  produce  a 
risk  value,  e.g..  High,  Moderate,  Low,  for  each  consideration. 
Since  the  schedules  are  generated  at  multiple  levels  of  detail, 
the  checklists  must  address  those  same  levels.  The  checklists 
are  now  reviewed  to  ensure  their  consistency  with  the  decision¬ 


making  criteria  originally  developed  in  the  PSP  (Step  1). 

After  finalizing  the  evaluating  checklists,  they  will  be 
used  to  review  each  concurrency  alternative.  These  checklists 
will  be  structured  to  address  the  activities  and  events  resched¬ 
uled  in  the  alternatives.  However,  in  applying  them,  the  values 
and  degree  of  concurrency  and  risk  determined  for  each  option  or 
group  of  activities  must  also  be  considered. 

In  the  third  substep,  the  projected  cost,  readiness,  and 
schedule  risks  associated  with  each  alternative  are  quantified. 
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The  result  of  this  analysis  is  a  ranking  according  to  cost, 
readiness  of,  and  schedule  risk  of  the  alternative  schedules. 

This  ranking  reflects  the  results  of  applying  the  checklists  to 
each  alternative,  in  light  of  the  following  considerations: 
degree  of  concurrency, 

total  risk  calculated  and  the  peak  risk  estimated, 

amount  of  uncertainty  associated  with  the  resource  and 
schedule  estimates, 

dependency  relationship  among  activities  and  events, 

overall  influence  of  activity  in  program  schedule  and 
cost, 

stage  of  system  technology  development,  and 

perceived  scope  of  impact  of  decision/consequences  of 
failure  of  schedule  or  cost  projections. 

Specific  attention  must  be  given  to  determining  the  risks  of 
exceeding  the: 

•  total  costs  if  the  concurrently  scheduled  activity 
fails  to  succeed. 


•  total  schedule  if  the  concurrently  scheduled  activity 
fails  to  succeed,  and 

•  projected  activity  cost  or  schedule  estimate. 

The  final  substep  in  the  risk  evaluation  portion  of  the 
analysis  involves  aggregating  the  risk  values  to  the  predeter¬ 
mined  decision-making  level  of  detail.  Depending  on  the  circum¬ 
stances  this  may  occur  at  the  Summary,  Detailed  or  Sub-schedule 
level . 

Step  7 ;  Select  New  Schedule 

Purpose:  To  make  decisions  on  schedule  revisions  based  on 

analyzing  risks  associated  with  the  alternatives. 
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Approach: 


r 


* 


7.1 

7.2 

7.3 

7.4 

7.5 

7.6 


Review  and  revise  decision-making  criteria  in  PSP 

Review  and  revise  proposed  schedule-monitoring  techniaues 

r-S-ltS  °£.risl1  analy*i*  of  alternatives  q 

^S10n"^aklng  criteria  to  Viable  alternatives 
Select  alternative 

Revise  existing  schedule 


The  final  step  in  this  analysis  involves  making  decisions  on 
the  alternative  schedules.  In  the  preceeding  steps  preliminary 
decisions  would  have  been  made  on  how  to  decide  which  of  the 
alternatives  will  be  selected  and  how  to  evaluate  the  effective¬ 


ness  of  the  revised  schedule.  The  first  substep  in  this  final 
part  of  the  analysis  is  to  review  and  revise,  if  necessary,  the 
decision  criteria  contained  in  the  PSP.  in  the  process  of 
identifying  and  reviewing  the  concurrency  options,  and  developing 
and  evaluating  the  alternative  schedules,  it  is  quite  possible 
that  additional  imperatives  contributing  to  the  decision-making 
process  will  be  identified.  The  criteria  should  be  modified  to 
incorporate  those  additional  considerations • 

In  addition  to  reviewing  the  decision-making  criteria  it  is, 
at  this  point,  also  useful  to  review  the  originally  proposed 
techniques  for  monitoring  the  revised  schedule  and  the  potential 

a^ternatj‘ves  *  set  of  alternative  schedules  some  will  be 

eliminated  for  future  consideration  simply  by  the  choice  of  a 
particular  alternative.  However,  some  alternatives  may  not  be 
totally  eliminated  as  possibilities  since  their  divergence  from 
the  revised  schedule  occurs  later  in  the  program.  These  alterna¬ 
tives  should  be  monitored  as  the  schedule  progresses  to  allow 
their  maintenance  as  scheduling  options. 
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The  third  substep  involves  analysis  of  the  results  of  the 
risk  analyses,  performed  in  Step  6.  The  risk  values  developed 
for  each  alternative  are  arrayed  on  a  graph  illustrating  their 
comparative  cost  and  schedule  risk  values.  Additional  illustra¬ 
tions  such  as  cost  and  schedule  contours  are  also  developed  as 
part  of  this  substep. 

The  fourth  substep  involves  evaluating  each  of  the  risk- 
assessed  alternatives  in  terms  of  the  decision-making  criteria. 

If  constructed  adequately,  these  criteria  represent  the  signifi¬ 
cant  points  of  concern  and  priorities  of  the  FM.  Each  alterna¬ 
tive  is  given  a  ranking  based  on  the  risk  assessment  and 
application  of  the  decision-making  criteria. 

The  fifth  substep  is  the  actual  selection  of  the  alternative 
or  revised  schedule,  and  the  secondary  alternatives  which  will  be 
monitored. 

The  final  substep  in  this  process  is  the  initiation  of  the 
revised  schedule  and  incorporation  of  it  into  the  program  plan. 
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CHAPTER  8.  ANALYSIS  OF  CONCURRENCY  SCHEDULE  RISK 

•  The  Role  of  Risk  Analysis  in  Acquisition  Planning 

•  Concurrency  Application  Considerations 

•  Evaluation  of  Risks 
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CHAPTER  8.  ANALYSIS  OF  CONCURRENCY  SCHEDULE  RISK 


* 


v 


In  the  preceeding  chapter,  a  structured  approach  for  analyz¬ 
ing  concurrency  impacts  and  revising  schedules  was  described. 

Part  of  this  analysis  involves  developing  risk  targets  and  cal¬ 
culating  estimates  of  risk.  In  this  chapter,  potential  applica¬ 
tions  of  this  risk  analysis  are  discussed.  (Detailed  discussions 

f  i 

of  the  elements  of  risk  analysis  and  actual  techniques  for  per¬ 
forming  risk  analysis  are  discussed  in  Part  IV  of  this  handbook, 
Chapters  9  and  10.) 

THE  ROLE  OF  RISK  ANALYSIS  IN  ACQUISITION  PLANNING 

Risk  analysis  should  have  an  important  place  in  the  planning 
and  execution  of  any  complex  program.  The  importance  that  must 
be  placed  on  the  performance  of  risk  analysis  is  enhanced  when 
concurrency  is  adopted  as  an  acquisition  or  management  strategy. 
The  principal  reason  for  this  is  that  the  ensuing  compression  of 
the  acquisition  process  as  a  result  of  concurrent  scheduling 
amplifies  the  negative  impacts  of  factors  that  introduce  risk  and 
uncertainty  into  program  planning.  These  factors  include: 

•  technical  problems  that  may  introduce  delays  in  the 
program  that  can  only  be  rectified  at  increased  cost  or 
sacrifice  of  performance  (system  readiness); 

•  pressure  applied  by  organizations  outside  the  Program 
Office  where  decisions  made  for  political  reasons  may 
have  a  significant  impact  on  the  program; 

•  assumptions  made  about  economic  conditions,  particu¬ 
larly  forecasts  such  as  those  of  inflation  rates,  that 
impact  program  cost  estimates; 
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changes  in  the  requirements  statement  as  a  result  of 

l^f™8?8811161^  °f  the  threat  or  changes  in  the  opera- 
na  environment  of  the  system  being  acquired;  and 


changes  in  the  acquisition  strategy  that 
from  changes  in  program  budget,  changes 
tion  schedule,  or  changes  to  the  planned 
and  support  philosophies  for  the  system. 


may  result 
in  the  produc- 
maintenance 


In  considering  the  use  of  concurrency  as  a  scheduling 
option,  it  is  important  to  be  able  to  analyze  the  risks  asso 
ciated  with  that  implementation  as  well  as  with  alternative 


scheduling  options.  In  this  regard,  development  factors  such  as 
design  status,  familiarity  of  technology,  environmental  charac¬ 
teristics,  project  personnel  experience,  contractor  availa¬ 


bility/experience,  and  production  factors  such  as  production 
resource  availability/manufacturing  capability,  and  level  of 
previous  program  involvement  are  all  important.  But  so,  too,  is 
the  discipline  required  to  schedule  far  in  advance  of  actual 
requirements,  e.g.,  to  consider  production  and  logistics  problems 
very  early  in  the  cycle.  There  is  a  complex  hierarchy  of  respon¬ 
sibility  and  review  that  also  contributes  to  the  problem  rather 
than  to  the  solution.  m  addition  to  these  needs,  the  program 
schedule  must  also  be  analyzed  in  terms  of  its  sensitivity  to 
external  forces  such  as  political  and  budgetary  decisions. 

In  assessing  the  impacts  of  concurrent  scheduling,  the 
Program  Office  must: 


identify  the  risks  that  are  introduced. 


determine  what  advanced  planning  is  necessary, 


decide  on  the  precedence  relationships  that  exist 
the  various  projects  and  tasks  to  be  performed. 


among 
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*  produced!*6  Critical  ?ath  any  schedule  that  is 

•  locate  the  "choke  points"  in  the  schedule,  and 

a^f ixed^amount  Sf  “mlTor* 'dona^f ^  take 

of  the  level  of  effort  appUed  L  themf0" 

These  are  just  some  of  the  particular  aspects  of  planning  for 

concurrency  that  must  be  considered.  Others  !are  identified 

throughout  this  handbook. 

The  last  item  listed  above  is  an  important  one  since  it  is 
often  overlooked  in  planning.  As  an  example  of  planning  consid¬ 
erations  that  must  be  made  in  this  area,  it  is  instructive  to 
discuss  the  development  of  avionics  software.  Exhibit  8-1  illus¬ 
trates  the  general  time  span  for  avionics  software  development. 
Among  the  precedence  relationships  that  occur  in  program  plann¬ 
ing.  is  one  that  comes  about  due  to  the  fact  that  avionics  soft¬ 
ware  development  cannot  be  started  until  system  requirements  are 
known.  Usually  this  has  not  been  before  FSD  (Full-Scale  Develop¬ 
ment).  The  first  operational  flight  program  usually  takes  from 
36  to  42  months  to  develop,  unless  nuclear  certification  is 

required,  in  which  case  12  months  must  be  added  to  the  planntng 
time  ' 

The  reason  that  avionics  development  time  is  relatively 
fixed,  is  that  it  is  a  people-intensive,  very  sequential 
activity.  And  as  with  most  software  development  programs,  adding 
more  coders  would  not  shorten  the  development  time  and  may  only 
add  inefficiency.  That  is  coupled  with  the  fact  that  upfront, 

V  ?ontro!SBranchS(IlD/M)e?„°d  th1  e,?perience  °£  the  Avionics 

Branch  (ASD/AX)  m  developing  avionics  systems  software. 
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Exhibit  8-1.  AVIONICS  SOFTWARE  DEVELOPMENT  SCHEDULE 
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very  detailed  design  is  necessary  for  this  component,  since  every 
pound  of  avionics  added  to  an  aircraft  will  increase  the  total 
aircraft  weight  by  4  to  10  pounds,  depending  on  its  placement  on. 
the  craft.  Taking  shortcuts  in  this  area  can  be  very  costly 
since  requirements  for  power  and  cooling,  in  addition  to  aircraft 
weight  and  other  design  considerations,  are  affected.  While 
these  considerations  apply  to  avionics  software,  similar  situa¬ 
tions  occur  for  many  other  systems  in  specific  weapon  systems. 
These  must  be  identified  in  developing  the  program  schedules, 
particularly  in  applying  concurrency. 

When  examining  the  use  of  concurrency  as  an  acquisition 
stategy,  one  should  consider  the  two  relevant  planning  concepts: 

•  time-constrained  scheduling,  and 

•  resource-constrained  scheduling. 

In  time-constrained  scheduling,  strategies  would  focus  on  alter¬ 
native  allocations  of  resources  and  sequencing  of  work  that 
accomplishes  a  project  in  a  fixed  amount  of  time.  Resource- 
constrained  scheduling,  on  the  other  hand,  is  performed  when 
resources  (generally  budgets,  personnel,  or  equipment)  are 
limited.  In  this  case,  one  generally  attempts  to  accomplish  a 
project  in  minimum  time  within  the  resource  limit. 

Two  criteria  that  are  frequently  used  to  measure  the  effec¬ 
tiveness  of  schedules  are  measures  of  project  slippage  and 
resource  utilization.  The  first  is  used  since  there  are  gener— 
ally  penalty  costs  associated  with  not  completing  a  project  on 
time.  Those  penalties  may  be  posed  as  lost  “good  will"  with 
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users  of  the  product  or  reviewing  authorities;  or  it  may  be  posed 
as  an  opportunity  cost  resulting  from  missed  interfaces  with 
other  projects  or  plans.  Resource  utilization  is  an  important 
measure  of  program  efficiency  since  idle  resources  are  costly. 

One  measure  of  resource  utilization  often  used  is  the  ratio  of 
the  total  resources  scheduled  in  a  period  of  time  to  the  total 
resources  available  during  that  period. 

Both  time-constrained  scheduling  and  resource-constained 
scheduling  can  be  dealt  with  using  standard  scheduling  techni¬ 
ques.  Often,  however,  such  techniques  as  PERT/CPM  (see  Chapter 
10)  are  not  appropriate  for  resource  constrained  scheduling, 
especially  with  very  large  and  complex  projects.  That  is  because 
resource  constraints  cannot  be  automatically  accommodated  by 
the  technique.  As  a  result,  if  a  postulated  schedule  violates  a 
resource  constraint,  it  may  be  time-consuming  and  costly  to 
re-schedule  using  PERT/CPM. 

Concurrency  can  be  used  as  a  strategy  for  scheduling  in  a 
time-constrained  situation,  either  for  an  entire  program  or  for 
individual  projects.  The  simple  notion  behind  this  strategy, 
when  used  in  a  time-constrained  situation,  is  to  schedule  all 
that  can  be  scheduled  in  each  time  period.  One  has  to  be  careful 
in  this  instance,  as  with  all  scheduling,  but  particularly  here, 
not  to  violate  precedence  relationships  that  may  exist  among  the 
various  projects  or  tasks  that  have  to  be  performed.  In  addi¬ 
tion,  although  it  may  be  feasible,  and  desirable,  to  interrupt 
tasks  for  periods  of  time,  one  must  be  careful  not  to  interrupt 
them  at  critical  or  infeasible  points.  With  these  thoughts  in 
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mind  it  is  useful  to  look  more  closely  at  the  analysis  of  risks 
in  conjunction  with  the  use  of  concurrency  and  some  considera¬ 
tions  involved  in  such  an  application. 


CONCURRENCY  APPLICATION  CONSIDERATIONS 


As  has  been  mentioned  throughout  this  handbook,  the  concept 
of  concurrency  can  be  applied  in  many  different  ways,  for  many 
different  reasons.  The  following  quote  elaborates  on  this.  While 
it  is  discussing  concurrency  in  Navy  ship  acquisitions,  the  same 
applies  to  Air  Force  programs,  as  witnessed  by  the  F-16 
multistage  improvement  program. 

Concurrency  is  often  used  in  two  areas  of  the  acquisition 
process  to  minimize  the  time  required  to  acquire  a  class  of 
ships.  First,  there  may  be  concurrency  in  the  development  of  the 
detail  design  and  early  construction  efforts.  Normally  it  is 
expected  that  subsystems  or  equipments  will  be  tested  and 
accepted  for  fleet  use  before  they  are  designated  for  inclusion 
in  a  new  class  of  ships.  However,  situations  may  arise  in  which 
major  improvements  are  anticipated  from  equipment  currently  under 
development.  In  those  cases,  a  decision  must  be  made  on  whether 
accept  the  lower  performance  available  from  proven  equipment 
or  to  accept  some  risk  by  continuing  development  of  new  equip¬ 
ments  that  promise  to  meet  the  projected  performance  goals  and 
completion  schedule. 

"The  second  aspect  of  concurrency  is  ordering  the  early 
follow  ships  before  the  first  ship  has  been  delivered  and  tested. 

The  decision  on  the  timing  of  the  award  for  early  follow  ships 
and  start  of  construction  of  these  ships  should  reflect  a  trade¬ 
off  between  an  acceptable  level  of  risk  that  the  lead  ship  will 
satisfy  the  stated  requirements  and  the  desire  to  deliver  follow 
ships  as  early  as  possible  so  that  they  will  have  maximum  useful 
^•^e*  is  because  of  the  latter  reason  that  decisions  are  made 

on  most  ship  acquisition  programs  to  not  utilize  the  lead  ship  as 
a  true  prototype  for  the  remaining  ships  of  the  class  .  Another 
reason  for  rejecting  the  prototype  approach  is  that  the  ship 
system  as  a  whole  generally  incorporates  hardware  which  is  off- 
the-shelf,  state-of-the-art  and  therefore  does  not  pose  the  kind 

-  Much  of  the  following  discussion  has  been  adapted  from  Shortening 
the  Acquisition  Cycle;  Research  on  Concurrency  (I ns ley,  et  al. 
Management  Consulting  &  Research,  Inc.,  30  September  1982. ) 
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of  risk  posed  by  a  system  comprised 
advanced  — technology  hardware 

Decisions  on  the  use  and  placement  of 

tingent  upon  several  conditions: 


for  the  most  part  of 
concurrency  are  con- 


•  the  magnitude  of  the  constraint,  i.e.,  the  amount  of 
time  that  the  schedule  must  be  reduced? 

•  the  portion  of  the  schedule  affected  by  the  constraint, 
i*e.,  the  activities  remaining  to  be  accomplished  or 
which  can  be  rescheduled;  and 

•  the  opportunity  to  analyze  the  risks  and  impacts  of 
making  the  decision. 

t; 

This  analysis  should  be  part  of  the  overall  effort  to 
develop  and  update  the  acquisition  plan/strategy.  This  means 
that  the  concurrency  analysis  should  be  initiated  in  the  Concept 
Exploration  phase  and  identified  in  the  outline  of  the  acquisi¬ 
tion  plan/strategy.  There  are  two  reasons  for  advocating  early 
concurrency  analysis: 

•  the  earlier  the  process  is  incorporated,  the  earlier 
the  alternatives  and  risks  can  be  identified  and  moni- 
tored;  and 

•  the  initial  analysis  may  be  complex,  however,  once  the 
apparatus  for  performing  the  analysis  has  been  devel¬ 
oped,  subsequent  reviews  will  be  easier  to  implement. 

Early  planning  also  allows  the  gathering  of  the  necessary 
information  and  organization  of  the  schedule  to  facilitate  the 
concurrency  analysis  from  the  beginning.  This  is  particularly 
critical  due  to  the  need  to  identify  tasks  or  activities  which 
are  analytically  compatible.  The  concurrency  analysis  rests  on 
the  ability  of  the  analysts  to  determine  how  much  of  an  activity 


_  Relationship  Between  Acquisition  Strategy  and  the  Contract 
Design  Package,"  Advanced  Marine  Enterprises,  Inc.,  Arlinqton, 
Virginia,  22  February  1977. 
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is  complete,  or  will  be  complete,  at  a  given  point  in  time.-' 

This  is  used  in  the  calculation  of  the  degree  of  dependence  the 
activity  has  on  other  activities,  combined  with  the  degree  of 
uncertainty  related  to  the  resource  projections.  The  degree  of 
uncertainty,  in  turn,  is  related  to  how  much  of  the  activity  has 
actually  been  completed  at  the  time  of  the  analysis,  when  the 
activity  occurs  in  the  sequence,  how  dependent  the  activity  is  on 
other  activities,  and  how  sensitive  it  is  to  exogenous  factors. 
Exhibit  8-2  illustrates  the  degree  of  technological  uncertainty 
at  progressive  stages  of  the  acquisition  process.  A  similar 
pattern  exists  in  the  accomplishment  of  activities  within  these 
stages.  It  may,  however,  be  more  useful  to  the  PM  to  measure 
activities  not  in  terms  of  amount  of  work  completed  but  rather  by 
the  completion  of  the  amount  of  time  allocated  for  the  task.  The 
only  requirement  is  that  whatever  measure  is  used  allows  for  a 
meaningful  comparison  of  the  tasks. 

The  ultimate  goal  of  the  analysis  is  to  provide  the  decision 
maker  with  a  method  for: 

•  identifying  activities  which  can  be  concurrently  sched¬ 
uled,  and 

•  evaluating  the  cost,  readiness  and  schedule  risks 

associated  with  them.  i 

/ 

order  to  do  this  the  analysts  must  ask  a  series  of  qualitative 
questions  which  assist  in  determining  the: 

•  degree  of  activity  dependence, 

1/  Many  program  management  guides  describe  standard  techniques  for 
analyzing  work  completed  or  programs.  A  brief  discussion  of  this 
is  given  in  the  Guide  for  Weapon  Systems  Acquisition,  (Huffman, 
Lozito,  and  Snyder,"  Air  Command  and  Staff  College,  May  1981). 
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•  amount  of  acceptable  concurrency  to  be  permitted  among 
different  activities,  and 

•  amount  of  acceptable  cost  and  schedule  risk  considered 
tolerable  for  the  planned  scope  of  the  concurrency.  ' 

Questions  which  should  be  answered  are,  for  example: 

•  What  information  is  needed  to  begin  the  activity? 

•  What  are  the  sources  of  this  information? 

•  Are  they  under  the  control  of  the  PM? 

•  How  much  of  the  activity  has  been  completed  at  this 
time? 

•  How  much  of  the  tasks  which  provide  input  information 
to  this  activity  must  be  complete  before  this  activity 
can  be  initiated? 

•  Is  the  source  activity  (the  activity  or  task  providing 
information),  expected  to  meet  its  schedule?  If  not, 
how  uncertain  is  this  schedule? 

The  analysis  should  use  two  different  checklists.  These 
checklists  are  to  be  used  to: 

•  evaluate  activities  and  events  to  determine  concurrency 
options,  and 

•  evaluate  the  cost,  readiness,  and  schedule  risks  asso¬ 
ciated  with  the  different  schedule  alternatives. 

In  identifying  the  concurrency  options,  activities  are  initially 

categorized  in  terms  of  those  that: 

•  cannot  be  rescheduled,  reorganized  or  reordered; 

•  might  be  possible  to  reschedule,  reorganize  or  reorder, 
but  for  a  variety  of  reasons  are  less  desirable;  and 

•  can  be  rescheduled,  reorganized  or  reordered. 

Initial  emphasis  would  be  placed  on  the  third  category. 

Assignment  to  any  of  the  categories  is  based  on  current  under¬ 
standing  of  the  conditions  prevailing  in  the  program.  It  is. 
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therefore,  possible  that  activities  previously  considered  as 
unlikely  concurrency  options  may,  after  further  consideration,  be 
re-categorized .  As  mentioned  in  Chapter  7,  the  identification  of 
potential  concurrency  options  is  of  substantial  importance  since 

those  options  provide  the  basis  for  generating  the  alternative 
schedules . 


EVALUATION  OF  RISKS 

*  r  ' 

The  concurrency  options  are  grouped  in  various  combinations  - 
and  with  different  sets  of  constraints  in  order  to  generate 
different  schedules.  An  option  comprises  at  least  a  pair  of 
activities,  composed  of  an  independent  or  source  activity,  i.e., 
the  activity  which  provides  information  to  the  dependent  activ¬ 
ity,  and  the  dependent  activity  which  succeeds  the  source 
activity  and  would  begin  before  completion  of  the  source 
activity.  Options  may  actually  comprise  clusters  of  activity/ 
event  combinations,  representing  all  of  the  sub-schedule  activi¬ 
ties  related  to  a  detailed  schedule  activity.  The  composition  of 
an  option  is  completely  dependent  upon  the  perceived  requirements 

of  the  project.  However,  generally,  dependent  activities  within 
an  option  should  be: 

•  under  the  control  or  direction  of  the  PM,  and 

•  influence  the  project  cost  or  schedule  duration. 

Exhibit  8-3  illustrates  the  basic  relationship  of  concur¬ 
rency  options  to  alternative  schedules.  A  sequence  of  activi¬ 
ties,  Tasks  A  through  F,  is  shown.  Based  on  analysis  of  the 
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I.  Aicfcpti&l*  Dl|ni  *£  Cwitttrr*ncy  4TUl  Rlak 


Option  Degree  of  Concurrency 

A-B  •  B  +  5  Months/37. 5X 

C-D  .  a  C  +  4  Month»/66.7X 

a  E  +  3  Months/25 , OX 

C/E-F  a  F  +  2  Months/33,31 


Acceptable  Target 
Degree  of  Coat  Risk* 

-10 

-30 

•05 

.20 


Acceptable  Target 
Degree  of  Schedule  Risk* 

.15 
.20 
•  10 
•  15 


*  Degree  of  Risk  -  Probability  of  failure  to  neet  estisuited  coat  or  schedule. 


C.  Alternative  Schedule 


TAG  If. 


Exhibit  8-3.  RELATIONSHIP  OF  CONCURRENCY 
OPTIONS  TO  ALTERNATIVE  SCHEDULES 
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dependency  relationships,  the  degree  of  uncertainty  associated 
with  them,  and  the  role  of  each  task  in  the  total  sequence, 
options  are  selected.  The  options,  any  or  all  of  which  may  be 
used  in  generating  the  alternatives,  are: 

•  overlap  of  B  with  A,  A-B  option; 

•  overlap  of  C  with  D,  C-D  option; 

•  overlap  of  E  with  D,  D-E  option;  and 

•  overlap  of  F  with  both  C  and  E,  C/E-F  option. 

Having  determined  the  options,  limiting  values  must  be 

developed  to  use  in  generating  the  alternative  schedules.  The 
characteristics  which  are  manipulated  for  each  alternative  are: 

•  the  applicable  concurrency  options, 

•  t*Z  !^at!d  ^®sou^Ge  estimates  for  each  activity  on  the 
schedule  (reflecting  the  additional  constraints), 

•  the  maximum  acceptable  degree  of  concurrency  for  each 
option,  and 

•  the  maximum  acceptable  degree  of  cost  and  schedule  risk 
tor  each  option. 

Alternatives  are  structured  taking  into  account  the  degree 
of  uncertainty  associated  with  the  initial  estimates.  Part  C  of 
Exhibit  8-3  shows  the  alternative  generated,  based  on  the  selec¬ 
tion  of  options  and  tailoring  of  values  for  the  characteristics. 

After  generating  the  various  alternatives,  it  is  necessary 
to  evaluate  the  risks  of  each  in  terms  of  cost  and  schedule.  The 
basic  tools  in  this  analysis  are  the  cost  risk  evaluation  check¬ 
lists  and  the  schedule  risk  evaluation  checklists.  They  are 
designed  to  address  the  concerns  .the  decision  maker  must  keep  in 
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mind  in  order  to  weigh  the  alternatives.  Exhibits  8-4  and  8-5 
are  examples  of  the  kinds  of  considerations  necessary  for  devel¬ 
opment  of  the  checklists.  Actual  checklists  would  have  to  be 
tailored  to  the  particular  application  at  hand.  The  purpose  of 
the  risk  evaluation  is  not  only  to  estimate  the  potential  risk 
associated  with  each  alternative,  but  also  to  rank  the  appro- 
priateness  of  each  alternative.  It  is  possible  to  generate 
al ternatives  with  mutually  conflicting  cost  and  schedule  con¬ 
straints  or  risks. 

Exhibit  8-6  illustrates  the  results  of  evaluating  the  alter¬ 
natives  generated  in  the  example  given  in  Exhibit  8-3.  In  addi¬ 
tion  to  calculating  the  total  amount  of  time  saved,  the  specific 
cost  and  schedule  tasks  must  be  calculated  for  each  option  as 
well  as  the  total  for  the  alternative.  As  part  of  this  analysis, 
it  is  also  important  to  determine  the  "peak"  risk,  i.e.,  the 
options  with  the  highest  potential  cost  and  schedule  risk.  This 
is  particularly  important  if  the  potential  risk  is  higher,  or 
related  to  a  different  option  than  the  original  "target"  degree 
of  risk,  as  illustrated  in  this  example. 

The  effectiveness  of  the  application  of  the  concurrency 
analysis  methodology  can  be  quantified  once  this  analysis  has 
been  made.  Suppose  that  the  portion  of  interest  in  the  schedule 
for  a  particular  project  cannot  be  completed  in  less  than  T0 
units  of  time,  say  months.  Suppose  further  that  the  baseline 
schedule  for  that  portion  of  the  project  has  a  duration  of  Tg 
months.  If  a  concurrent  schedule  will  complete  the  project  in  Tc 
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•  Amount  of  "Farm-out" 
of  Design  Effort 


•  Number  of 
Subsystems 


•  Timing  of  Events/ 
Schedules 


•  Number  of 
Interfaces 


•  Level  of  Experience 
with  System  Type 


•  Size  &  Type 
of  System 


•  Externally  Imposed 
Constraints 


•  Development 
of  Hardware 


•  Externally  Imposed 
Changes  in  Require¬ 
ments 
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of  Hardware 
and  Software 
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•  Tech.  Support 

Management  Plans 
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Exhibit  8-4.  COST  RISK 


CONSIDERATIONS  IN  SYSTEM  DESIGN 


Is  the  activity  dependent  on  information  or  inputs  from 
outside  the  performing  organization? 

Is  the  manpower  within  the  performing  organization 
subject  to  fluctuation;  i.e.,  is  the  performing  organi¬ 
zation  responsible  for  performing  similar  activities 
for  other  programs  and,  therefore,  the  manpower  must  be 
competed  for? 


Does  the  ability  to  compete  (or  lack  thereof) 
indicate  relative  value  or  importance  and,  there¬ 
fore,  can  the  program  expect  to  have  lower 
priority  in  other  areas? 

How  much  of  the  input  information  is  needed  in  order  to 
begin  the  dependent  activity? 

Is  development  of  the  input  information  from  outside  of 
the  performing  organization  on  time?  Do  they  expect  it 
to  stay  on  time? 

What  other  parts  of  the  schedule  are  dependent  on  this 
information?  Information  from  this  group? 

Has  allowance  for  schedule  slippage  been  incorporated 
in  the  time  estimate? 


Have  additional  quality  assurance  measures  been  identi¬ 
fied  in  order  to  reduce  potential  risks  associated  with 
concurrently  scheduling  the  activity? 


Exhibit  8-5.  SCHEDULE  RISK  CONSIDERATIONS 
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Calculated  Degree  of  Risk  for  Alternate* 


Option  Cost  Risk 

A-B  20%  (10%) 

C-D  30%  (30%) 

D-E  5%  (5%) 

C/E-F  35%  (20%) 


Scheduled  Risk 

15%  (15%) 
20%  (20%) 
15%  (10%) 
40%  (15%) 


Position 

Highest  Target  Cost  &  Schedule  Risk 

Highest  Potential  Risk 
Highest  Potential  Schedule  Risk 


* 


(%) 


Acceptable  Target  of  Degree  of  Risk  for  Option 


Results  of  Evaluation  of  Alternative  A 

Effect  of  Concurrency:  Gain  5  Months 
Total  Cost  Risk: 

Total  Scheduled  Risk: 

Peak  Cost  Risk: 

Peak  Scheduled  Risk: 
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Exhibit  8-6.  EVALUATION  OF  ALTERNATIVE  SCHEDULE 


months,  then  one  measure  of  the  effectiveness  of  the  concurrency 
accomplished  by  the  latter  schedule  is: 


Clearly,  this  is  a  relative  measure  since,  for  any  given 
project,  the  potential  effectiveness  of  applying  concurrency  will 
change  as  the  completion  time  for  the  baseline  schedule  changes 
and  the  schedule  progresses.  Simply  stated,  this  measure  of 
the  degree  of  concurrency  gives  the  percent  of  time  that  can  be 
saved  in  the  baseline  schedule  that  is  actually  saved  by 
implementing  the  concurrent  schedule. 

Generally  speaking,  projects  are  not  completed  in  the  short¬ 
est  possible  time,  e.g.,  in  Tg  years.  That  is  because  of  budget 
limitations  or  the  risks,  technological  and  otherwise,  that  are 
introduced  as  one  tries  to  shorten  the  acquisition  time.  Thus, 
the  degree  of  concurrency  sought,  Dc,  must  be  balanced  against 
the  risk  of  successful  program  completion  within  a  specified  bud¬ 
get  and  time,  and  producing  a  specified  level  of  product  performance 
In  Exhibit  8-7  Dq  is  plotted,  for  varying  levels  of  TQ/TB, 
against  the  term  (TB-TC)/TB.  As  used  here,  (TB-TC/TB)  is  a  meas¬ 
ure  of  the  percent  of  the  time  it  takes  to  complete  the  baseline 
schedule  that  is  saved  by  implementing  the  schedule  with  .concurrency 
In  selecting  the  alternative  which  will  be  used  to  revise 
the  existing  schedule,  the  decision  maker  must  take  into  con¬ 
sideration  the  other  scheduling  alternatives  which  can  be  used  in 
conjunction  with  concurrency. 
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Some  of  the  alternatives  the  decision  maker  (or  PM)  should 
consider  are: 

•  funding  of  parallel  activities,  in  order  to  increase 
the  probability  that  one  of  the  alternatives  will 
successfully  meet  the  goals  of  the  program; 

•  funding  repetition  of  activities,  when  a  critical 
activity  has  not  been  previously  successful; 

•  scheduling  activity  "slack  time,"  to  allow  for  the 
unforeseen  extension  of  the  duration  of  an  activity; 
and 

•  lowering  performance  objectives  of  a  high-risk  activity 
and  compensating  by  increasing  the  performance  require¬ 
ment  for  a  lower  risk  activity. 

The  ultimate  set  of  decisions  made  by  the  PM  must  reflect  the 

particular  needs  of  the  project. 


PART  IV.  OVERVIEW  OF  RISK  ANALYSIS 

Chapter  9.  Elements  of  Risk  Analysis 
Chapter  10 .  Risk  Analysis  Alternatives 
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PART  IV.  OVERVIEW  OF  RISK  ANALYSIS 

One  of  the  major  concerns  in  this  handbook  is  the  considera¬ 
tion  of  how  to  best  manage  risks  to  materiel  readiness  in  devel- 
oping  a  new  system.  In  order  to  effectively  consider  this,  it  is 
necessary  to  consider  the  actual  nature  of  risk  analysis,  how  it 
is  conducted  and  the  role  it  plays  in  program  management.  The 
latter  is  discussed  throughout  this  handbook.  The  chapters  in 
this  section  of  the  handbook  focus  on  the  nature  of  risk  and  the 
elements  of  risk  analysis  ( in  Chapter  9 ) ,  and  some  of  the  tools 
and  alternatives  available  to  the  Program  Manager  in  examining 
risks  (Chapter  10). 
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CHAPTER  9.  ELEMENTS  OF  RISK  ANALYSIS 


The  Requirement  for  Risk  Analysis 
The  Nature  of  Risk  and  Uncertainty 
Phases  of  Risk  Analysis 


CHAPTER  9.  ELEMENTS  OF  RISK  ANALYSIS 
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The  four  principal  elements  in  the  acquisition  of  a  major 
system  are  program  cost,  the  development  and  production  schedule 
of  the  program,  the  performance  that  the  system  will  achieve,  and 
the  supportabil ity  and  readiness  requirements  of  the  system. 
Effectively  balancing  these  four  elements  becomes  increasingly 
difficult  as  systems  become  more  costly  and  more  complex  technol¬ 
ogically,  and  their  introduction  into  the  force  becomes  more 
time— cr i t i ca 1 .  In  attempting  to  balance  the  four  elements  of 

cost,  schedule,  performance,  and  readiness,  DoD  planners  and 
managers  have  instituted  a  series  of  initiatives  that  address 
various  aspects  of  the  acquisition  process.  One  set  of  those 
initiatives  addresses,  among  other  things,  the  need  to  consider 
sifstnative  acquisition  strategies  and  the  resulting  requirement 
to  evaluate  the  effectiveness  of  those  strategies.  Among  the 
reasons  for  this  requirement  has  been  the  tendency  toward  cost 
growth  and  schedule  slippage  on  many  military  acquisition  programs. 

This  chapter  contains  a  review  of  selected  guidance  requir¬ 
ing  the  performance  of  risk  analysis  in  acquisition  programs,  and 
covers  aspects  of  risk  analysis  that  include: 

•  relevant  definitions  and  an  explanation  of  the  distinc¬ 
tion  between  risk  analysis  and  uncertainty  analysis, 

•  the  role  of  risk  analysis  in  acquisition  planning,  and 

•  analytic  considerations  that  must  be  addressed  in  per¬ 
forming  a  risk  analysis. 
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THE  REQUIREMENT  FOR  RISK  ANALYSIS 


On  31  July  1969,  then  Deputy  Secretary  of  Defense  David 
Packard  instructed  each  of  the  Service  Secretaries  to  ensure 
that,  during  concept  formulation:  "areas  of  high  risk  are 
identified  and  fully  considered;  formal  risk  analysis  on  each 
program  is  made;  and  summaries  of  these  are  made  part  of  the 
backup  material  for  the  program."^  in  September  1969, 
then  Secretary  of  the  Air  Force  Robert  Seamans  noted  the 


following : 


lac.  ^f,ti;L1  anKtheE  significant  reason  for  cost  growth  in  the 
last  few  years  has  been  the  failure  to  adequately  identify  the 
risks  associated  with  major  programs.  This  should  occur  early  in 
the  project,  definition  phase.  Late  recognition  of  significant 
uncertainties  can  be  disastrously  expensive.  In  the  future,  we 
will  make  a  formal  risk  analysis  of  each  of  our  programs.  We  must 
guard  against  the  combination  of  optimistic  pressures,  includinq 
our  own  eagerness  to  get  on  with  the  job. "1/ 


During  the  1970s,  directives  and  implementing  instructions 
formalized  the  risk  assessment  requirement  for  major  system 
acquisitions.  On  5  April  1976,  the  Office  of  Federal  Procurement 
Policy  within  the  Office  of  Management  and  Budget  (OMB)  issued 
0MB  Circular  No.  A-109,  Major  Systems  Acquisitions.  That  docu¬ 
ment  states  a  requirement  to: 


perform  coordinated  mission  planning  that  results  in 
requirements  stated  in  terms  of  mission  needs  rather 
than  hardware  specifications, 

examine  alternative  procurement  actions  for  meeting  the 
stated  mission  needs,  and 

develop  acquisition  strategies  that  are  tailored  to 
individual  programs. 


Lenox,  Hamilton  T.,  Risk  Assessment,  Air  Force 
nology,  Wright-Patterson  AFB ,  Ohio,  June  1973. 


Institute  of  Tech- 
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As  part  of  the  acquisition  Strategy  for  new  acquisitions,  the 
circular  also  requires  an  evaluation  of  the  risks  involved  in  the 
acquisition. 

DoD  Directive  5000.1,  Major  System  Acquisitions  (29  March 
1982),  states  that  the  phases  of  the  acquisition  process  for  a 
new  system  "are  to  be  tailored  to  fit  each  program  to  minimize 
acquisition  time  and  cost,  consistent  with  the  need  and  the 
degree  of  technical  risk  involved."  Moreover,  that  directive 
states  that  the  decision  to  procede  to  full-scale  development  may 
be  delayed  "until  some  additional  development  effort  has  been 
accomplished  to  provide  better  definition  of  performance,  cost, 
schedule ,,  producibil ity,  industrial  base  responsiveness,  support- 
ability,  and  testing  to  reduce  risk  and  uncertainty  before  the 
commitment  to  a  major  increase  in  the  application  of  resources 
toward  full-scale  development  is  made." 

On  31  March  1982,  then  Deputy  Secretary  of  Defense  Carlucci 
outlined  32  initiatives  (the  "Carlucci  Initiatives")  planned  to 
improve  the  system  acquisition  process  within  DoD.  Of  the  32 
initiatives,  seven  directly  -influence  the  use  of  concurrency  as 
an  acquisition  strategy  and  one,  Action  11  -  Technological  Risk 
Funding ,  directly  addressed  risk  in  the  acquisition  process. 

In  DoD  Instruction  5000.2,  Major  System  Acquisition 
Procedures  (8  March  1983),  the  specification  for  the  System 
Concept  Paper  (prepared  for  Milestone  I)  requires  that  the  System 
Program  Office  clearly  identify  those  key  areas  of  technological 
risk  that  have  to  be  reduced  before  the  Milestone  II  decision. 
That  same  instruction  requires  that  the  Decision  Coordinating 
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Paper  (prepared  for  Milestone  II )  explain  how  test  and  evaluation 
results  indicate  resolution  of  significant  risks. 

DoD  Directive  5000.39,  Acquisition  and  Management  of 
Integrated  Logistic  Support  for  Systems  and  Equipment 
(17  November  1983),  states  that  acquisition  strategies  will 
emphasize  "evaluation  of  alternative  support  concepts  and 
techniques  to  minimize  cost  and  support  risks." 

Finally,  MIL-STD-1388-1A,  Logistic  Support  Analysis 
(11  April  1983),  provides  guidance  for  identifying  early  require¬ 
ments  for  analyses  that  can  be  performed  to  obtain  a  balance 
among  cost,  schedule,  performance,  and  supportability . 

Risk  analysis,  as  an  aid  in  analyzing  acquisition  strategies 
is  thus  a  necessary  part  of  the  analyses  performed  during  the 
course  of  the  system  acquisition  process. 

THE  NATURE  OF  RISK  AND  UNCERTAINTY 

Generally  speaking,  there  are  three  states  of  knowledge  into 
which  a  given  circumstance  may  be  classified.  The  circumstance 
may  involve  "knowns, "  i.e.,  facts  and  issues  for  which  no  further 
resolution  is  necessary.  It  may  involve  "known-unknowns,"  i.e., 
elements  which  are  known  to  require  resolution,  but  for  which 
further  analysis  is  necessary  or  certain  assumptions  must  be 
made.  Finally,  the  circumstance  may  involve  what  are  called 
"unknown-unknowns,"  i.e.,  unanticipated  situations  that  cannot  be 
predicted  before  they  occur.  The  latter  include  random  events 
and  externalities  over  which  one  has  no  control. 
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It  is  useful  at  this  point  to  consider  examples  of  the 
second  and  third  states  of  knowledge  since  they  help  form  the 
basis  of  definitions  of  risk  and  uncertainty.  An  example  of  a 
situation  that  involves  "known-unknowns"  is  the  fabrication  of  a 
wing  for  a  new  airplane  using  conventional  design  and  manufactur¬ 
ing  techniques.  Although  there  are  a  number  of  problems  that 
would  have  to  be  resolved  here,  they  are  well  known  and  have  been 
dealt  with  many  times  before.  "Unknown-unknowns,"  on  the  other 
hand,  were  the  cause  of  the  serious  instability  experienced  with 
early  large  liquid  fuel  rockets.  The  cause  was  eventually  deter¬ 
mined  to  be  fuel  sloshing."  This  had  not  been  anticipated  since 
there  was  no  prior  experience  in  this  area  and  analysis  had  not 
predicted  this  phenomenon.  Only  after  the  loss  of  several 
rockets  was  the  cause  determined.”^ 

Although  the  terms  risk  and  uncertainty  are  often  used 
interchangeably,  strictly  speaking,  they  are  quite  different 
technically.  The  following  are  accepted  definitions  and  are  the 
focus  of  theoretical  work  and  work  on  analytical  techniques: 

•  Certainty  is  a  term  used  to  describe  situations  in 
which  each  action  is  known  to  lead  invariably  to  a 
specific  outcome. 

•  Risk  enters  in  when  each  action  leads  to  one  of  a  set 
of  possible  outcomes,  each  outcome  occurring  with  a 
known  probability. 

•  Uncertainty  is  present  when  each  action  has  as  its 
consequence  a  set  of  possible  specific  outcomes,  but 


U  George,  B.  M. ,  "The  Weapon  Systems  Development  Process,"  Govern- 

ment  Procurement  and  Contracting  (Part  9).  Hearings  Befori~i - 

Subcommittee  of  the  Committee  on  Government  Operations,  House  of 
Representative3,  Ninety-first  Congress,  First  Session  on  H.  R. 
474,  1969.  ' 
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the  probabilities  of  those  outcomes  are  either 
completely  unknown  or  are  not  meaningful.  3/ 

There  are  many  factors  that  may  lead  to  risk  and  uncertainty 
in  an  acquisition  program.  Some  of  those  factors  come  about  due 
to  the  complexity  of  the  decision  making  that  must  be  done. 
Typically  a  Program  Manager  must  make  decisions  in  an  environment 
that  imposes  multiple,  Sometimes  conflicting,  objectives,  and 
often  it.  will  be  difficult  to  even  identify,  let  along  choose 
among,  alternative  courses  of  action.  The  conflicting  objectives 
come  about  for  a  number  of  reasons,  including  the  presence  of 
several  decision  makers  to  whom  the  Project  Manager  is  responsi¬ 
ble  and  must  respond,  and  a  system  operational  environment  that 
! 

includes  many  impacted  groups. 

Uncertainty  in  the  decision  making  process  enters  as  a 
result  of  many  intangibles  that  are  generated  by  the  particular 
technologies  involved,  the  program  planning  and  weapon  system 
operational  environments  involved,  data  uncertainty,  requirements 
for  subjective  judgement,  and  trade-offs  among  alternatives. 
Unpredictable  disruptions  that  cause  a  gap  in  program  execution 
may  occur  in  a  program,  resulting  in  a  requirement  to  stretch  a 
program  over  a  longer  time  for  lack  of  funding  or  because  key 
milestones  could  not  be  met.  Even  uncertainty  in  aspects  of 
models  used  for  analysis  can  have  an  effect  if  unpredicted 
results  arise  due  to  model  misspecif i cat  ion  or  data  uncertainty. 

Not  the  least  measure  of  uncertainty  is  imposed  by 

unanswered  (or  unanswerable)  questions  concerning  the  threat  to 

1/  Luce,  R.  D.  and  Raiffa,  H.,  Games  and  Decisions,  John  Wiley, 
1957;  see  also  Final  Report  of  the  USAF  Academy  Risk  Analysis 
Study  Team,  Aeronautical  Systems  Division,  August  1971. 
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which  a  particular  system  is  designed  to  respond.  Rapid  technological 
advances  and  changing  operational  environments  make  it  particu¬ 
larly  difficult  to  do  accurate  long-range  planning  without 
accepting  some  jeopardy  to  the  program  cost  or  schedule,  or  to 
weapon  system  performance.  These  factors  can  materially  affect 
the  successful  performance  of  an  acquisition  program. 


PHASES  OF  RISK  ANALYSIS 


In  order  to  maK©  F©©§@n§fele  acquisi£i©R  management  deci¬ 
sions,  a  risk  analysis  following  the  structure  displayed  in 


Exhibits  9-1  gnd  9-2  might  be  performed.  T'ne  approaches  that  are 
illustrated  are  fairly  general  and  are  readily  tailored  to 
individual  problems.  Using  them  to  structure  an  analysis  should 
prove  useful. 

Conceptually,  at  least,  the  approacnes  that  are  illustrated 
can  be  thought  of  as  consisting  of  four  phases: 

•  Phase  I  -  Problem  Identification, 

•  Phase  II  -  Problem  Formulation, 

•  Phase  III  -  Analysis,  and 

•  Phase  IV  -  Evaluation. 

Each  of  these  phases  is  discussed  below  and  specific  elements  of 
these  phases  are  explained  in  the  following  subsections.  These 
phases  are  labeled  on  Exhibit  9—1  to  enhance  the  discussion. 

Because  of  the  nature  of  the  graphic  display  in  Exhibit  9-2,  it 
is  more  difficult  to  clearly  identify  each  phase  without  compli¬ 
cating  tne  illustration.  In  addition,  although  the  impression  one  . 
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Source : 


L.  W.  ,  A  Risk  Management  Model  for  the  Defense  System 
Acquisition  Process;  SWL,  Inc . ,  McLean ,  Virginia  (undated) . 


Exhibit  9-1. 


A  SAMPLE  STRUCTURE 


FOR  A  RISK 


ASSESSMENT 
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Source:  Kraemer,  George  T.,  A  Successful  Quantitative  Risk  Assessment 

Technique ,  Boeing  Vertol  Company  (undated) 


Exhibit  9-2.  A  SAMPLE  RISK  ASSESSMENT  FLOW  DIAGRAM 
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gets  from  the  two  exhibits  and  the  following  discussion  may  be 
that  this  construct  is  designed  for  large  risk  analysis  problems 
only,  it  applies  equally  well  to  small  problems. 

1 •  Phase  I  -  Problem  Identification 

The  first  phase  of  the  risk  assessment  structure 
involves  clearly  identifying  the  problem  to  be  analyzed  and  the 
work  that  has  to  be  carried  out.  The  importance  of  this  step 
should  be  obvious.  It  is  during  this  phase  that  one  should  begin 
to  think  about  the  scope  of  the  problem  as  well  as  the  ramifica¬ 
tions  of  potential  decisions.  The  latter  is  of  particular  impor¬ 
tance  at  this  point  since,  in  order  to  adequately  perform  a  risk 
assessment,  one  must  fully  account  for: 

•  all  of  the  decisions  that  must  be  made, 

•  all  of  the  possible  outcomes  of  those  decisions, 

•  the  impacts  of  the  decisions  on  other  groups,  and 

•  the  resulting  ramifications  of  those  impacts. 

That  is  not  to  say  that  exhaustive  analyses  of  each  of  these 
should  be  performed,  but  each  topic  should  be  given  some  attention. 

The  product  of  this  phase  of  the  risk  assessment  should 
be  an  easily  understood  description  of  the  problem  to  be 
addressed,  a  statement  of  the  goals  of  the  analyses  to  be  per¬ 
formed,  and  at  least  the  beginning  of  a  framework  for  conducting 
those  analyses. 

2 •  Phase  II  -  Problem  Formulation 

In  the  problem  formulation  phase,  one  must  fully 
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describe  the  problem(s)  at  hand  and  decide  in  very  precise  terms 
exactly  what  is  desired  by  way  of  analysis.  It  is  in  this  phase 
that  conceptualization  of  the  problem  occurs  insofar  as  analyti¬ 
cal  constructs  for  treating  the  problem  are  suggested.  If,  for 
instance,  the  problem  is  one  of  developing  a  master  program 
schedule,  this  phase  of  tljie  work  would  involve  steps  designed  to 
identify : 

•  those  elements  of  the  program  to  be  covered  by  the 
project  schedule, 

•  the  techniques  to  be  applied  in  formulating  and  dis¬ 
playing  the  schedule, 

•  the  particular  analyses  to  be  performed  to  resolve 
specified  issues,  and 

•  the  analysis  teams  to  carry  out  the  next  phase  of  the 
assessment  structure. 

In  the  particular  example  cited,  i.e.,  developing  a 
master  program  schedule,  the  first  step  would  result  in  specifi¬ 
cation  of  those  aspects  of  the  program  to  be  considered  and  the 
level  of  detail  to  which  each  aspect  will  be  included.  The  vari¬ 
ous  levels  that  could  be  considered  include  the  following  hier¬ 
archy  of  acquisition  program  components: 

•  Phases  -  The  major  acquisition  phases,  as  defined  by 
DoD  Directive  5000.1,  are:  Concept  Exploration, 
Demonstration  and  Validation,  Full-Scale  Development, 
and  Production. 

•  Functions  -  The  major  categories  of  work  performed  in, 
or  under  the  direction  of,  the  Program  Office,  include 
Technical  Management,  Logistics  Management,  Business 
Management,  and  other  general  categories  of  work 
activity  and  Project  Office  responsibility. 

•  Task  Areas  -  These  are  the  subtasks  of  functional  work 
such  as  those  that  come  under  Technical  Management, 
i.e.,  hardware  design,  software  design,  test  and  evalu¬ 
ation  and  others. 
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•  Events  -  These  are  the  key  points  in  a  Task  Area  that 
identify  the  beginning  or  end  of  some  segment  of  work. 
Examples  are  document  delivery,  design  review  meetings, 
milestones,  and  initiation  of  document  preparation. 

•  .  •  .  ' .  .  i 

•  Activities  -  Many  individual  efforts  are  often  involved 
in  order  to  achieve  satisfactory  accomplishment  of  an 
event.  Examples  of  these  efforts,  which  involve 
preparation  for  a  particular  ending  event,  or  which 
follow  a  starting  event,  include  preparation  of  a  base¬ 
line  schedule,  and  review  of  a  procurement  plan. 

•  Organizations  -  There  are  often  many  organizations 
involved  in  the  work  of  the  Program  Office.  These 
groups  are  responsible  for  performing  activities  as 
either  Program  Office  functional  groups  or  contractors. 

Each  of  the  six  components  listed  above  plays  a  critical  role  in 

the  planning  and  analysis  of  the  Program  Office.  Each  represents 

a  different  level  of  detail  at  which  program  planning  can  be 

done . 


Once  the  appropriate  level  of  detail  is  determined,  the 
range  of.  techniques  that  can  be  brought  to  bear  on  the  problem 
should  be  considered.  The  actual  selection  of  a  technique  should 
consider: 

•  the  resources  required  to  implement  the  technique,  i.e., 

personnel, 
time,  and 
cost; 

•  the  availability  of  data  needed  to  support  the  alterna¬ 
tive  techniques;  and 

•  the  robustness  of  the  technique,  i.e.,  the  sensitivity 
of  the  solution,  as  a  function  of  the  technique,  to 
changes  in  parameters  including: 

-  data, 

-  particular  algorithms  used. 
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precision  of  the  computations,  and 
user  group. 

As  a  result  of  the  problem  identification  phase,  there 
should  be  a  cone ise  . statement  of  the  problem  to  be  treated. 

Implicit  in  the  statement  of  the  problem  is  a  set  of  questions 
that  must  be  answered  or  at  least  addressed.  The  analyses  that 
will  be  performed,  given  the  level  of  detail  and  the  techniques 
chosen,  must  be  targeted  at  the  particular  questions  raised. 

Once  the  analyses  are  decided  upon,  one  can  assemble  the  indivi¬ 
duals  most  suited  to  solving  the  problem  at  hand. 

Once  problem  formulation  is  complete,  analyses  may  begin. 

3.  Phase  III  -  Analysis 

The  analysis,  or  implementation  phase,  is  the  phase  in 
which  the  chosen  techniques  are  used  to  answer  the  questions 
raised.  In  this  phase,  attention  should  be  paid  to  the  level  of 
detail  deemed  appropriate  and  to  the  accuracy  of  the  data  used. 

Because  of  the  possibility  of  pressures  imposed  from 
outside  the  Program  Office,  earlier  decisions  may  have  to  be 
re-evaluated  during  this  phase.  As  a  result,  alterations  in  the 
actual  implementation  may  have  to  be  made. 

4 .  Phase  IV  -  Evaluation 

No  analytic  structure  is  complete  without  an  evaluation 
phase.  During  this  phase  one  should  consider: 

•  the  consistency  of  the  analysis  results  with  what  is 

already  known  or  highly  suspected, 
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•  the  direct  impact  of  those  results  on  the  project,  and 

•  the  ramifications  of  the  results  on  the  interfaces  that 
the  Program  Office  must  have  with  other  organizations. 

Once  this  is  done,  additional  analyses  may  be  warranted  or  the 

task  may  be  deemed  complete. 

The  phases  of  the  risk  assessment  structure  have  been 
categorized  differently  by  a  number  of  individuals.  One  instruc- 
tive  way  to  group  the  various  activities  involved  is  shown  in 
Exhibit  9—3.  That  exhibit  not  only  poses  the  activities  slightly 
ly,  but  also  introduces  a  number  of  terms  that  relate  to 
the  different  components  of  analysis  of  risk. 

Risk  analysis  is  the  general  process  we  have  been  dis¬ 
cussing  in  this  section.  As  seen  from  the  discussion  above,  it 
is  the  process  of  examining  the  probabilities  and  consequences  of 
the  outcomes  of  a  set  of  decisions.  Risk  analysis  is  typically 
used  to  assess  the  degree  to  which  a  proposed  system  is  likely  to 
achieve  its  predicted  performance  within  cost  and  schedule  goals. 
The  overriding  objective  of  risk  analysis  is  risk  reduction  or  at 
least  risk  management.  Risk  management  refers  to  those  actions 
implemented  with  the  expressed  purpose  of  reducing  the  number  of 
program  factors  at  risk  or  reducing  the  level  of  risk  for  those 
factors .  The  factors  that  are  at  risk  are  identified  by  way  of 
procedures  and  analyses  that  are  referred  to  as  risk  assessment. 

The  next  chapter  of  this  report  describes  a  number  of 
techniques  that  have  been  successfully  used  in  performing  risk 
analyses  in  acquisition  programs  for  the  Department  of  Defense. 
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Source : 


Defense  Systems 
A  Handbook  for 


Colle9fe/  Risk  Assessment  Techniques: 
Program  Management  Personnel,  July  1983 .  , - * 
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Exhibit  9-3.  COMPONENTS  OF  RISK  ANALYSIS 


CHAPTER  10.  RISK  ANALYSIS  ALTERNATIVES 

•  Network  Techniques 

•  Simulation  Methods 

•  Graphic  and  Analytical  Methods 

•  Data  Requirements 


CHAPTER  10.  RISK  ANALYSIS  ALTERNATIVES 

Most  of  the  techniques  that  are  generally  applied  in  risk 
analyses  can  be  classified  into  the  following  categories: 

•  network  techniques,  including 

PERT  (Program  Evaluation  and  Review  Technique), 

VERT  (Venture  Evaluation  and  Review  Technique), 

f  | 

CPM  (Critical  Path  Method), 

TRACE  (Total  Risk  Assessing  Cost  Estimate),  and 
RISKNET; 

•  simulation  methods,  such  as 

RADSIM  (Research  and  Development  Assessment 
Model ) ,  and 

WBS  (Work  Breakdown  Structure)  Simulation; 

•  graphics  techniques  like  Gantt  Charting;  and 

•  analytical  methods  such  as 

CER  (cost  estimating  relationship)  development, 
cost/benefit  analysis,  and 
decision  analysis. 

These  are  just  some  of  the  many  tools  available  for  analyzing  the 
risks  associated  with  the  acquisition  of  major  systems. 

Since  there  are  quite  a  few  references  on  risk  analysis  that 
are  readily  available  (see  Appendix  A),  this  chapter  only  con¬ 
tains  a  brief  overview  of  risk  analysis  techniques.  More  discus¬ 
sion  of  particular  risk  analysis  techniques  may  be  found  in  the 
references  listed  in  Appendix  A,  especially  the  reference  Risk 
Assessment  Techniques: - A  Handbook  for  Program  Management  Personnel 

by  the  Defense  Systems  Management  College. 
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NETWORK  TECHNIQUES 


Network  techniques  have  come  to  be  widely  used  in  program 
management  as  a  way  to  plan  and  evaluate  the  schedule  for  accomp¬ 
lishment  of  program  activities.  Many  of  the  applications  in  use 
today  have  evolved  from  the  successful  application  of  a  technique 
called  PERT  (Program  Evaluation  and  Review  Technique)  on  the 
Polaris  Submarine  Program.  * 

PERT  is  especially  useful  on  complex,  multi-activity  pro¬ 
grams.  In  addition  to  providing  an  analytic  structure  for  organ¬ 
izing  and  scheduling  program  activities,  PERT  analysis  provides 
information  that  is  helpful  for  evaluating  program  schedules.  It 
allows  identification  of  the  critical  paths  among  program  activ¬ 
ities,  as  well  as  the  shortest  time  and  least-cost  paths.  PERT 
can  also  be  used  to  determine  the  probability  of  completing  the 
entire  program,  or  some  segment  of  it,  and  so  is  very  useful  for 
risk  assessment . 

Statistical  PERT  is  a  variation  of  PERT  that  involves  sim¬ 
plification  of  a  network  by  partitioning  it  and  modeling  subsets 
of  it  separately.  This  technique  is  used  on  programs  where  very 
large  networks  are  required  to  adequately  represent  program 
activities.  it  can  also  be  used  to  perform  in-depth  analysis  of 
program  modules  for  which  a  separate  scheduling  network  can  be 
devised . 

The  Critical  Path  Method  (CPM)  is  similar  to  PERT,  however 
it  does  not  use  simulation,  and  emphasizes  time/cost  trade-offs, 
whereas  PERT  generally  treats  these  parameters  individually.  As 
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with  all  network  techniques,  CPM  requires  that  the  program  activ¬ 
ities  be  expressed  in  terms  of  nodes  (milestones,  tasks,  or 
decision  points)  and  arcs  (tasks  or  resource  requirements). 

There  is  standard  software  that  permits  quick  solution  of  prob¬ 
lems  set  up  for  the  use  of  CPM,  and,  depending  on  the  size  of  the 
application,  CPM  can  be  used  on  micro-,  mini^,  and  mainframe 

I 

computers . 

VERT,  which  is  applicable  to  programs  where  performance  is 
measured  in  terms  of  time  or  cost,  offers  the  capability  of 
modeling  performance  values  for  each  activity  singly  or  jointly. 
As  with  most  network  simulation  methods,  application  of  VERT  can 
require  substantial  amounts  of  data.  There  is  considerable  flex¬ 
ibility  with  VERT,  however,  in  that  14  probability  distributions 
are  embedded  in  VERT  computer  software  and  can  be  used  in  model¬ 
ing  a  program  schedule. 

TRACE-P  (Total  Risk  Assessing  Cost  Estimate  for  Production) 
is  an  integration  of  VERT  and  WBS  (work  breakdown  structure) 
techniques.  This  technique  is  used  for  risk  assessment/contin- 
gency  funding  in  the  first  few  years  of  system  production.  The 
essential  innovation  in  this  technique  is  that  a  direct  link  is 
made  between  WBS  elements  and  the  production  schedule. 

RISKNET  has  been  used  in  a  variety  of  complex  programs  with 
multiple  activities  and  milestones  where  it  is  desirable  to 
express  total  program  risk  in  terms  of  cost  or  time  requirements. 
RISKNET  requires  a  mainframe  computer  for  its  operation.  It 
allows  the  user  to  perform  sensitivity  analyses  and  provides  an 
opportunity  to  model  stochastic  elements  of  program  scheduling. 
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SIMULATION  METHODS 
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RADSIM  and  WBS  Simulation  are  two  techniques  that  use  simu¬ 
lation  as  the  basis  for  evaluating  the  elements  of  a  program  that 
are  at  risk.  RADSIM,  which  is  similar  to  PERT,  analyzes  those 
data  that  describe  the  technical  expectations  of  all  R&D  activi¬ 
ties  that  are  potential  problem  areas.  In  order  to  do  that,  the 
user  must  specify  the  logic  by  which  the  program  is  structured 
and  the  data  that  describe  the  strategy  by  which  the  program  is 
to  be  conducted.  WBS  Simulation,  as  its  name  implies,  uses  a 
work  breakdown  structure  in  generating  a  total  program  cost  esti¬ 
mate.  Individual  cost  estimates  for  elements  of  the  WBS  are 
described  by  probability  distributions.  The  total  program  cost 
is  then  derived  probabilistically  as  the  sum  of  the  various 
(probabilistic)  component  costs. 

There  are  a  number  of  general  purpose  simulation  languages 
that  can  be  used  in  risk  analyses.  These  include  CSMP  (Contin¬ 
uous  System  Modeling  Program SIMSCRIPT,^  GPSS  (General 

Purpose  System  Simulation),-7  and  GASP  (General  Activity  Simula- 
3  / 

tion  Program).— ' 


— 7  Gordon ,  G.,  System  Simulation, 
New  Jersey,  1969. 


Prentice-Hall , 


Englewood  Cliffs, 


2/ 


Wyman,  F.  P.,  Simulation  Modeling: 
John  Wiley  &  Sons,  New  York,  1970. 


A  Guide  to  Using  SIMSCRIPT, 


-7  Pritake,  A.  B. ,  and  Kiviat,  P.  j. , 
FORTRAN  Based  Simulation  Language, 
Cliffs,  New  Jersey,  1969. 


Simulation  with  GASP  II:  A 
Prentice  Hall,  Englewood 
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GRAPHIC  AND  ANALYTICAL  METHODS 

Graphical  methods  include  analysis  of  descriptive  statistics 
of  stochastic  variables  as  well  as  techniques  such  as  Gantt 
Charts  and  control  charts •  These  methods  provide  a  visual  dis¬ 
play  of  important  variables  (e.g.,  program  cost,  probability  of 
success  or  failure,  system  performance)  as  a  function  of  one  or 
two  (for  3— dimensional  plots)  parameters  such  as  time,  resources, 
and  system  parameters  (weight,  range,  reliability  factors,  etc.). 

Analytical  methods  include  such  techniques  as : 

•  CER  development, 

•  cost/benefit  analysis,  and 

•  decision  analysis. 

Each  of  these  can  play  a  role  in  the  methods  discussed  above  in 
that  they  can  be  used  to  estimate  the  value  of  parameters  needed 
in  the  other  methods .  For  instance  one  might  develop  a  CER  to 
estimate  a  particular  cost  element  needed  in  PERT  or  in  a  WBS 
Simulation. 

CER  development  usually  centers  around  an  application  of 
regression  analysis  to  determine  a  representation  of  the  cost  of 
something  (a  system,  subsystem,  component,  etc.)  as  a  function  of 
technical  parameters  such  as  weight,  power  input,  peformance, 
etc.  Since  regression  is  used  in  CER  development,  there  is  a 
rich  statistical  theory  that  can  be  brought  to  bear  to  determine 
confidence  intervals  for  the  estimates  made. 

Cost/benefit  analysis  generally  results  from  a  quantifica¬ 
tion  of  both  the  cost  of  a  particular  item  or  program  element  and 
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its  associated  benefit.  A  comparison  of  those  two  quantities  can 
provide  a  way  to  evaluate  the  wisdom  of  undertaking  an  activity. 
The  comparison  can  be  made  in  any  number  of  ways.  In  particular, 
if  one  takes  the  ratio  of  cost-to-benef it ,  then  values  of  that 
ratio  above  a  pre-selected  threshold  would  signal  a  decision 
against  the  activity,  and  values  of  the  ratio  below  the  thresh¬ 
old  would  favor  a  decision  to  perform  the  activity. 

Decision  analysis  is  a  general  category  of  analysis  that 
includes  methods  in  the  areas  of  stochastic  processes,  utility 
theory,  and  game  theory,  and  draws  on  such  fields  as  probability, 
statistics,  optimization,  and  economics  for  specific  techniques. 
The  general  approach  often  described  in  this  area,  although  many 
could  be  described,  is  one  of  analyzing  a  sequence  of  events.  The 
events  are  generally  hierarchical,  i.e.,  they  are  ordered  and 
some  subsets  of  events  occur  only  based  on  the  outcomes  of  pro- 
ceding  events.  When  probabilities  of  occurrence  are  associated 
with  each  event  and  the  events  are  independent,  one  can  easily 
determine  the  probability  of  any  feasible  sequence  of  the  events. 
That  information  can  then  be  used  for  decision  making.  For 
instance,  suppose  a  system  is  being  prepared  for  a  milestone  and 
a  decision  must  be  made  to  proceed  on  one  of  two  schedules.  The 
above  scheme  could  be  used  to  determine  the  probability  of  suc¬ 
cess  of  each  schedule  and  the  selection  could  be  made  with  that 
probability  as  one  input.  One  would,  of  course,  include  other 
information,  such  as  the  cost  of  each  schedule. 
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DATA  REQUIREMENTS 


Many  of  the  techniques  that  are  used  by  Program  Offices 
require  significant  amounts  of  data  to  execute.  That  is 
especially  true  of  the  scheduling  tecniques.  Apart  from  the 
shear  magnitude  of  the  data  requirement,  some  types  of  data 
elements  can  be  difficult  to  generate.  That  is  especially  true 
of  data  elements  that  represent  probabilities.  A  number  of 
techniques  have  been  developed  to  assist  in  this  area  when 
experimentation  is  infeasible  or  too  costly  or  when  sufficient 
"hard  data"  is  not  available. 

For  instance,  in  lieu  of  fully  specified  probability  distri 
butions,  PERT  uses  a  beta  distribution  and  estimates  of  the  opti¬ 
mistic  <ta),  most  likely  <tb>,  and  pessimistic  <tc)  time  to 
complete  an  activity.  The  expected  time  <te)  for  the  completion 
Of  that  activity  is  then  given  by  the  formula: 

t  4t,  ,  t 

f  _  a  +  o  +  c 

e  6 

Another  method  for  generating  probability  distributions  is 
the  Delphi  technique.  There,  expert  opinion  is  gathered  and 
through  a  structured  questionnaire,  one  can  estimate  the  proba¬ 
bility  of  events  by  polling  experts  in  the  field.  Simulation  is 
yet  another  technique  that  can  be  used  to  generate  needed  proba¬ 
bilities.  However,  this  method  can  itself  require  considerable 

effort  to  utilize  due  to  its  own  requirements  for  data  and  pro- 
gramming. 
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Often,  historical  data  can  be  used  to  project  future  values 
of  parameters.  Regression  and  time  series  analyses  are  often- 
used  analytical  techniques  for  this  purpose.  Extrapolation  along 
a  regression  line  when  time  is  the  independent  variable  can  be  a 
useful  tool,  when  coupled  with  analysis  of  the  confidence  inter¬ 
vals  of  the  estimate.  When  there  is  seasonality  in  the  histori¬ 
cal  data,  i.e.,  when  there  are  several  distinguishable  trends  at 
play,  time  series  techniques  (e.g.,  the  Box-Jenkins  method) 
should  be  used. 

There  are  many  factors  to  consider  in  applying  risk  analysis 
techniques.  The  application  of  the  techniques  themselves  is  com¬ 
plicated  by  the  complexity  and  implications  of  Program  Office 
planning  and  decision-making,  and  by  the  validity  of  the  data 
available  to  support  the  analysis.  Therefore  the  analyses  must 
be  done  carefully;  they  must  be  well— planned,  accurately  exe¬ 
cuted,  and  systematically  evaluated. 

References  for  each  of  the  techniques  discussed  can  be  found 
in  Appendix  A. 
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PART  V.  PLANNING,  MANAGEMENT,  AND  ACQUISITION  STRATEGIES 
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The  final  part  of  this  handbook  addresses  the  various  stra¬ 
tegies  the  Program  Manager  and  his  staff  can  adopt  for  balancing 
materiel  readiness  risks  and  concurrency  in  acquisitions.  These 
suggestions  are  contained  in  Chapter  11,  SUGGESTIONS  ON  STRATE¬ 
GIES,  the  final  chapter  in  this  handbook. 
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•  Planning  Strategies 

•  Management  Strategies 

•  Acquisition  Strategies 
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CHAPTER  11.  SUGGESTIONS  ON  STRATEGIES 


The  ultimate  goal  of  the  PM  in  balancing  materiel  readiness 
risks  and  concurrency  in  an  acquisition  is  to  be  able  to  produce 
a  system  within  the  given  cost,  schedule,  performance  and  sup- 
portability  constraints.  The  approach  for  accomplishing  these 
goals  is  laid  out  initially  in  the  program' s  acquisition  stra¬ 
tegy.  The  acquisition  strategy  is  the  guidance  document 
identifying  the  program's  priorities. 

In  addition  to  developing  the  formal  acquisition  strategy, 
we  suggest  that  the  PM  formulate  associated  strategies  for  plan¬ 
ning  and  management.  These  strategies  elaborate  on  the  acquisi¬ 
tion  strategy  concepts  and  are  intended  to  clearly  communicate  to 
the  PO  staff  and  contractors  the  PM ’ s  intentions  for  implementing 
the  program  priorities. 

This  chapter  contains  general  suggestions  for  developing 
each  of  these  strategies. 

PLANNING  STRATEGIES 

In  programs  with  concurrency,  the  PM,  Program  Control  Direc¬ 
torate  staff,  functional  area  managers,  and  contractors  must  all 
be  committed  to  actively  addressing  the  planning  implications  of 
concurrency.  Many  of  these  have  been  discussed  in  Chapters  2,  7 
and  8 . 

The  primary  element  of  the  planning  strategy  should  be  the 
recognition  that  tasks  and  activities  should  be  ’broken  down 
(decomposed)  to  the  lowest  level  of  effective  detail.  Too  often 
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in  programs,  activities  are  planned  and  monitored  at  such  an 
aggregate  level  that  vital  information  is  not  influential  in 
decision  making.  The  trade  off,  however,  must  be  made  between 
t"oo  much  and  too  little  data,  since  either  extreme  produces  the 
same  effect,  the  masking  of  vital  details. 

Using  the  priorities  set  out  in  the  acquisition  strategy, 
the  PM  and  functional  managers  should  translate  these  goals  into 
specific  functional  requirements  and  determine  the  specific  form, 
detail  and  process  for  tracking,  monitoring,  and  reporting  on 
the  status  of  these  priority— related  activities. 

In  LSA  planning,  logisticians  frequently  use  a  step-by-step 
program  sequencing  planning  approach.  This  approach  is  imple¬ 
mented  using  descending  levels  of  detail  in  each  of  the  ILS 
elements.  It  consists  of  interpreting  a  backward  and  forward 
sequence  of  activities,  (i.e.,  "If  I  have  to  be  able  to  have 
this,  then  this  must  be  done  first.")  Whatever  the  device  used, 
the  staff  must  have  a  clear  understanding  of  the  program  priori¬ 
ties,  with  sufficient  direction  to  be  able  to  make  decisions. 

The  planning  strategy  must  also  incorporate  a  clear  deter¬ 
mination  on  the  interactions  among  the  functional  groups  for 
supporting  these  priorities.  The  various  discussions  of  the 
readiness-related  activities  in  this  handbook  show  that  the 
efforts  of  many  functional  areas  are  involved.  These  interrela¬ 
tionships  and  dependencies  must  be  explicitly  recognized  and 
incorporated  in  any  planning  for  achieving  the  goal  of  high 
system  readiness. 
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Finally,  the  planning  strategy  must  recognize  the  need  to 
plan  for  contingencies.  The  following  statement  has  been  made 
regarding  the  development  of  schedules,  however,  it  applies  for 
all  planning  functions. 

"When  developing  schedules,  the  following  assumption  has  to 
be  made:  Nothing  works  right  the  first  time;  everything  goes 
wrong,  and  everybody  makes  mistakes . "i/ 

Once  again,  a  balance  must  be  reached  between  planning  for  every 
possible  incidence  of  Murphy's  Law  striking,  and  planning  with 
unrealistic  optimism.  The  planning  strategy  should  recognize  the 
high  risk  areas  of  the  program  and  encourage  the  development  of 
plans  for  constructing  recovery  programs,  should  the  need  arise. 
While  many  managers  reject  the  idea  of  planning  for  disasters,  a 
civil  defense  attitude  as  part  of  a  planning  strategy  may  encour¬ 
age  openness  in  dealing  with  the  inherent  problems  of  balancing 
materiel  readiness  and  concurrency. 

MANAGEMENT  STRATEGIES 

The  management  strategy  is  the  statement  of  the  philosophy 
and  approach  that  the  PO  intends  to  apply  and  expects  to  see 
implemented  throughout  the  program.  This  strategy  relates  to  how 
the  various  groups  involved  in  the  program  will  interact,  the 
hierarchy  of  authority,  and  the  processes  and  procedures  that 
will  be  used  to  manage  for  progress  while  minimizing  risks. 


_/  Godden,  Forrest  L.,  Jr.,  "Scheduling  for  Program  Management :  How 
and  Why, "  Concepts ,  Volume  4,  Number  4,  Defense  Systems  Manage¬ 
ment  College,  Autumn  1981. 
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A  key  act  of  the  management  strategy  is  the  management  con¬ 
trol  system  which  will  be  used  to  communicate  the  status  of  the 
activities.  The  following  is  a  suggested  set  of  criteria  for 
such  a  system:-' • 

•  furnish  timely,  pertinent,  adequate,  and  accurate 
information ; 

•  ensure  that  decisions  are  made  well  in  advance  of  per¬ 
formance  v it  must  force  effective  planning  before 
initiation  of  the  program); 

•  be  flexible  to  accommodate  necessary  changes; 

•  be  economical  in  operation; 

•  be  relatively  simple  in  operation  and  understandable  by 

all  users  of  the  system,  both  Government  and  contractor 
personnel ; 

•  permit  management  by  exception; 

•  provide  a  basis  to  evaluate  alternative  courses  of 
action  prior  to  initiation; 

•  forecast  trouble  areas  as  well  as  indicate  current 
deficiencies;  and 

•  indicate  significant  differences  between  planned  and 
actual  performances. 

In  addition  to  these,  the  management  system  must  have  an 
effective  "diagnostic”  system  in  it:  fault  isoloation  with  an 
associated  corrective  action  process.  There  ar6  many  success¬ 
fully  managed  programs,  and  a  feature  they  all  seem  to  share  is  a 
management  control  system  that  is: 

•  sensitive  to  the  critical  and  high-risk  areas; 

•  detailed  enough  to  respond  to  the  hidden  problems; 


— ^  — Programs: Program  Evaluation  and  Review  Techinnp 

AHcl  Ho.  U-6.  Army  Materiel  CommMdT' 3  January  1972^ 


(PERT ) , 
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frequently  exercised  enough  to  make  all  participants 
aware  of  its  operation; 

progress-oriented  so  as  to  not  allow  for  activities  to 
get  stalled  in  a  closed  loop  of  inactivity;  and 

organized  to  allow  for  consistent,  compatible  levels  of 
detail  to  be  developed  and  communicated  throughout  the 


program. 

If  these  sound  like  basic  management  practices,  it  should  not 
be  surprising.  Well  managed  programs  do  not  just  happen,  they 
must  be  deliberately  constructed  for  good  and  effective  manage¬ 
ment.  The  problems  of  balancing  materiel  readiness  and  concur¬ 
rency  are  such  that  such  a  management  strategy  is  virtually 

f 

mandatory . 


ACQUISITION  STRATEGIES 


Of  all  of  the  strategies  the  PM  must  develop,  the  acquisi¬ 
tion  strategy  is  probably  the  most  well  known.  This  strategy  is 
a  basic  requirement  in  program  management  and  must  be  approved 
through  the  chain  of  command. 

The  acquisition  strategy  is,  among  other  things,  the  state¬ 
ment  of  program  priorities.  As  such  it  is  the  pivotal  document 
for  clearly  stating  the  program  goals,  particularly  regarding 
readiness.  It  is  also  the  vehicle  which  can  be  effectively  used 
for  expressing  the  intended  applications  of  concurrency,  and  the 
circumstances  requiring  this  strategy.  If  the  use  of  concurrency 
is  intended  as  a  planning,  management,  or  acquisition  strategy, 
it  should  be  clearly  stated  in  the  acquisition  strategy. 
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The  acquisition  strategy  should  also  contain  a  clear  and 

definitive  exploration  of  how  the  concurrency  will  be  used  in 

conjunction  with  achieving  the  program's  readiness  goals.-  This 

means  including  clear  statements  of  how  these  goals  will  be 

represented  in  the  major  areas  of  program  activities.  The 

following  is  a  suggested  list  of  the  activities  that  should 
3  / 

be  discussed:-' 

•  Contracting: 

R&M  requirements, 

mission  profile  establishment, 

life  profile  establishment, 

R&M  failure  definition, 

-  incentives, 

source  selection  criteria,  and 

-  LCC  consideration; 

•  Management: 

planning,  control  and  emphasis  of  Government  and 
contractors, 

monitoring  and  control  of  subcontractors  and 
supplies,  and 

engineering  change  process; 

•  Design: 

development  of  design  requirements, 

design  alternative  studies, 

design  evaluation  analyses, 

parts  and  materiel  selection  and  control, 

-//  —.A/0SD .  J Reliability  and  Maintenance  Study,  Volume  III,  Case  Study 
Analysis,  Institute  for  Defense  Analyses,  November  1983. 


11-6 


derating  criteria, 

thermal  and  packaging  criteria, 

computer  aided  design, 

testability  analysis,  and 

testability  verification  and  testing; 

•  Production: 

environmental  stress  screening  (ESS),  and 
sysJems;randrtin9'  analysis  and  corrective  action 

•  Test  and  Evaluation: 

design-limited  qualification  testing, 
reliability  growth  testing, 
demonstration  testing, 
operational  test  and  evaluation,  and 
in-service  assessment. 

The  suggestions  presented  in  this  chapter  are  intended  to 
raise  thoughts  in  the  reader's  mind  regarding  the  interactions 
at  need  to  take  place  between  management  and  technical  func¬ 
tions,  and  to  sensitize  managers  to  the  readiness  implications  of 
concurrency  decisions. 
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This  appendix  contains  lists  of  instructions,  directives, 
regulations,  manuals,  standards,  and  studies  which  the  Program 
Manager  and  his  staff  should  review.'  The  lists  are  organized 
according  to  topic,  with  the  same  reference  possibly  appearing 
under  several  headings. 

This  is  not  intended  to  be  a  definitive  list  of  all  the 
relevant  documents  of  which  the  PM  should  be  aware.  Rather,  it 
is  an  initial  listing  which  should  be  used  as  the  basis  for  pur¬ 
suing  research  on  additional  sources  of  information.  These  lists 
have  been  constructed  from  a  variety  of  sources,  and  it  is  possi¬ 
ble  that  some  of  these  documents  are  no  longer  available,  may  be 
in  revision,  or  may  have  already  been  revised.  The  reader  should 
be  aware  of  this,  and  use  these  lists  as  a  guide. 
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ORGANIZATIONAL  RELATIONSHIPS  AND  RESPONSIBILITIES 


AFR  170-3,  Financial  Management  of  the  Security  Assistance 
Program.  — -  -  -  ■ 

AFR  800-2,  Acquisition  Program  Management. 

AFR  800-8,  Integrated  Logistics  Support  (ILS)  Program. 

AFLCR  800-9,  Management  of  DPML/lLSOs . 

AFLCR  400-1,  Logistics  Management  Policy. 

DOD  5105. 38M,  The  Military  Assistance  and  Sales  Manual. 

STATEMENT  OF  OPERATIONAL  NEED  (SON) /MISSION  ELEMENT  NEED 

STATEMENTS  (MENS! 

AFR  57-1,  Statement  of  Operational  Need  (SON). 

DOD  5105. 38M,  The  Military  Assistance  and  Sales»Manual . 

AFR  400-3,  Foreign  Military  Sales. 

AFLC  PROGRAM  ACTION  DIRECTIVE  (PAD)  AND  AFrC  PROGRAM  DIRECTION 

AFR  57-4,  Modification  Program  Approval. 

AFR  400-3,  Foreign  Military  Sales;  AFLC  Supplement  1. 

AFR  800-2,  Acquisition  Program  Management,  as  supplemented. 

AFSCR  27-1,  Program  Direction. 

AFLCR/AFSCR  57-3,  Class  V.  Modification  Management. 

AFLCR  57-21,  Modification  Program  Approval. 

AFLCR  400-1,  Logistics  Management  Policy. 

PLANNING,  PROGRAMMING,  BUDGETING  FOR  ACQUISITION  LOGISTICS  w 
AFM  172-1,  USAF  Budget  Manual. 

AFP  172-4,  The  Air  Force  Budget. 

AFLCR  27-3,  The  AFLC  Program  Objective  Memorandum  (POM). 

AFLCR  400-1,  Logistics  Management  Policy. 

United  States  Air  Force  Long  Range  Capabilities  Objective  (LRCO) . 
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The  Air  Force  Planning  Guide  (AFPG). 


MANPOWER  PLANNING  AND  ASSIGNMENTS 

AFM  26-1,  Manpower  Policies  and  Procedures. 

AFR  26-2,  Organization  Policy  and  Guidance. 

AFLCR  800-9,  Management  of  DPMLs/ILSOs. 

PROGRAM  MANAGEMENT  PLAN  (PMP) 

AFR  800-2,  Acgu isit ion  Program  Management. 

AFR  800-8,  Integrated  Logistics  Support  ( ILS )  Program  for  Systems 
and  Eguipment.  ”  ~  - - 

AFSCP  800-3,  A  Guide  for  Program  Management. 

INTEGRATED  LOGISTIC  SUPPORT  PLANS  (ILSP) 

AFR  800-8,  Integrated  Logistics  Support  (ILS)  Program,  as 
supplemented.  — 

INTEGRATED  SUPPORT  PLAN  (ISP) 

AF LCR/AFSCR  800-24,  Standard  Integrated  Support  Management 
System.  -  - a - 

LOGISTIC  SUPPORT  ANALYSIS  ( LSA ) 

M IL-STD-13 8 8- 1A,  Logistic  Support  Analysis. 

MIL-STD-1388-2 ,  Logistic  Support  Analysis  Data  Element 
Def in i t ions .  — 

M IL-STD-499 ,  Engineering  Management. 

AFSCM/AFLCM  800-4,  Optimum  Repair  Level  Analysis. 

CONTRACTING 

DOD  4105. 59H,  POD  Directory  of  Contract  Administration  Services 
Components .  ~  ~~  “ - 

DODI  4105.59,  DOD  Plant  Cognizance  Program. 

DODD  4105.62,  Selection  of  Contractual  Sources  for  Major  Defense 
Systems.  - 
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DODI  4105.64,  Technical  Representation  at  Contractors'  Facilities. 
DODD  5000.1,  Major  System  Acquisition. 

DODI  5000.2,  System  Acquisition  Process. 

DAR  ( AS P R )  1-406,  Contract  Administration  Functions. 

DAR  (AS PR)  1-2100.4,  Acquisition  Planning. 

AFSC  DAR  (ASPR)  Supplement  (C6),  20—703,  Retention  of  Contract 
Administration  by  the  Purchasing  Office. 

AFR  70-15,  Source  Selection  Policy  and  Procedures. 

AFSCR  80-15,  R&D  Source  Selection,  Policy  and  Guidance. 

AFSCR  70-7,  AFSC  Solicitation  Review  Panel. 

AFSCR  70-2,  Business  Strategy  Panel. 

DAR  (ASPR)  3-501,  Preparation  of  Request  for  Proposals  or  Request 
for  Quotations.  " 

INTERIM  CONTRACTOR  SUPPORT  ( ICS ) 

AFR  800-21,  Interim  Contractor  Support  for  Systems  and  Equipment 

AFM  172-1,  USAF  Budget  Manual. 

MIL-STD-499,  Engineering  Management. 

MIL-STD-1388-1A,  Logistic  Support  Analysis. 

MIL-STD-1388-2,  Logistic  Support  Analysis,  Data  Element 
Definitions . 

AFR  66-7,  Depot  Level  Maintenance  Production. 

WORK  BREAKDOWN  STRUCTURE  (WBS ) 

MIL-STD-881A,  Work  Breakdown  Structure  (WBS)  for  Defense  Material 
I  terns .  — —  ■  — — 

AFR  800-17,  Work  Breakdown  Structures  (WBS)  for  Defense  Material 
I  terns. 

AFLCP-AFSCP  173-5,  Cost/Schedule  Control  Systems  Criteria  (Joint 
Implementation  Guide). 
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RELIABILITY  AND  MAINTAINABILITY  (R&M) 


AFR  80-5,  Air  Force  Reliability  and  Maintainability  Program. 

MIL-STD-470,  Reliability  Program  for  Systems  and  Equipment, 
Development,  and  Production. 

MIL-STD-470,  Maintainability  Program  Requirement  (for  System  and 
Equipment ) . 

DODD  5000.1,  Major  System  Acquisitions. 

*  t 

DODI  5000.2,  Major  System  Acquisition  Procedures. 

SURVIVABILITY 

AFR  80-38,  Management  of  the  Air  Force  Survivability  Program. 
AFSC  Supplement  1. 

The  Management  of  Nuclear  Hardened  Parts,  Final  Report, 

AFALD/AQI  Project  77-14. 

Hardness  Awareness  Seminar,  TR-PTE/SV-78- 101 . 

B-l  Hardness  Assurance  Guidelines,  ASD-TR-75-35 . 

Nuclear  Hardness  Assurance  Guidelines  for  Systems  with  Moderate 

Requirements"  AFWL-TR-76-147  .  “ 

Missile-X  Program  Logistics  Element  Management  Plan  for  Nuclear 
Hardness  and  Survivability  Interface  LEM,  15  Aug.  77,  SAMSO/ 
MNL ,  AFSC. 

AFR  800-8,  Integrated  Logistics  Support  (ILS)  Program,  as 
supp lemented . 

LIFE  CYCLE  COST  (LCC)  MANAGEMENT  PROGRAM 
DODD  5000-28,  Design  to  Cost. 

AFR  66-14,  Equipment  Maintenance  Policies,  Objectives,  and 
Responsibilities. 

AFR  80-5,  Air  Force  Reliability  and  Maintainability  Program. 

AFR  800-11,  Life  Cycle  Cost  Management  Program. 

AFLCP/AFSCP  800-19,  Joint  Desiqn-To-Cost  Guide. 

AFP  173-13,  USAF  Cost  and  Planning  Factors  Guide. 
MIL-STD-1388-1A,  Logistic  Support  Analysis. 
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REPORTS 


AFM  67-1,  Volume  IX,  Chapter  7,  USAF  Supply  Manual. 

AFM  177-112,  International  Accounting  Transactions. 

AFR  400-3/AFLC  Sup  1,  Foreign  Military  Sales. 

AFLCR  800-25 ,  Command  Review  of  Systems  Acquisition  Programs. 

PROVISIONING  STRATEGY 

t  ' 

AFR  800-24 ,  Parts  Control  Program  (PCP). 

AFR  800-21,  Interim  Contractor  Support  for  Systems  and  Equipment. 
AFLCR  57-27,  Initial  Requirements  Determination. 

MIL-STD-965,  Parts  Control  Program. 

SUPPLY  SUPPORT 

AFR  57-6,  POD  High  Dollar  Spare  Parts  Breakout  Program. 

AFR  65-2,  Provisioning  of  End  Items  of  Ma^riel. 

AFR  66-1,  Maintenance  Management ♦ 

AFR  67-47 ,  Phased  Provisioning. 

AFR  80-5,  Air  Force  Reliability  and  Maintainability  Program. 

AFR  800-21,  Interim  Contractor  Support  for  Systems  Equipment. 

AFR  800-24 ,  Parts  Control  Program  (PCP). 

AFR  800-26,  Spares  Acquisition  Integrated  with  Production  (SAIP). 
AFLCR  57-4,  Recoverable  Item  Requirements  Systems  (D041). 

AFLCR  57—27 ,  Initial  Requirements  Determination. 

AFLCR  65-5,  Air  Force  Provisioning  Policies  and  Procedures. 

AFLCR  67-7 ,  Stock  Fund  Initial  Spares  Requirements . 

AFLCM/AFSCM  800-4,  Optimum  Repair-Level  Analysis  (ORLA). 

AFLCP  57-13,  Recoverable  Inventory  Control  Using  MOD-METRIC. 
MIL-STD-965,  Parts  Control  Program. 

MIL-STD-1517 ,  Phased  Provisioning. 
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MIL-STD-1552,  Provisioning  Technical  Documentation. 

MIL-STD-1561 ,  Provisioning  Procedures. 

MAINTENANCE  ACTIVATION  PLANNING 
AFk  8-2,  Air  Force  Technical  Order  System. 

AFR  50-9,  Special  Training. 

,  and 

AFR  86-1,  Programmin^civil  Engineer  Resources. 

AFR  89-1'  Design  and  Construction  Management. 

A™  172-1,  USAF  Budget  Manual. 

AFR  800-3,  Engineering  for  Defense  Systems. 

AFR  800-8,  Integrated _Log is tics  Support  (ils)  Program. 

AFR  800-12,  Acguisiti on  of  Support  Eguipment. 

AFR  800_21 '  factor  Support  for  systems  and  Eguignent 

^ -interservice  of 

AFLCR  66-17,  Dgpot  Maintenance  Support  Planning. 
™^S1»TT;”“tiCal  DePot  Industrial 

Maintenan«  source  of  Repair  (S0R,  Decision 

AFpSi^agea6^aI;m~r!!-Naine  ASSiqnment  Procedures  for  Type 

AFLCR  523-1,  Mission  Assignment  Policy. 

AFLCR  523-3,  AFLC  Mission  Assignment. 

AFSCP  800-3,  AGuide  for  Program  Management. 

AFSCR  800-10,  AFSC  Lessons  Learned  Program. 

^ *( S AT AF )J?R  800-11 '  g.i_te  Activation/Alteration  Task  Forces 
AFLCR/AFSCR  800-24,  Standard  Integrated  Support  Management  System. 


A- 6 


> 


AF LCR/AFSCR  800-30,  Maintenance  Interser vicing  New  Start 
Identification. 

MIL-STD-155,  Joint  Photographic  Type  Designation  System. 

MIL-STD-196,  Joint  Electronics  Type  Designation  System. 

MIL-STD-875,  Type  Designation  for  Aeronautical  and  Support 
Equipment . 

MIL-STD-1388-1A,  Logistic  Support  Analysis. 

H2-1,  2,  and  3,  POD  Cataloging  Handbooks. 

DEPOT  MAINTENANCE  INTERSERVICING  (DMI) 

AFLCR/AFSCR  800-24,  Standard  Integrated  Support  Management  System. 
AFLCR/AFSCR  800-30,  Depot  Maintenance  Inter  servicing. 

GOVERNMENT-FURNISHED  EQUIPMENT  (GFE) 

AFR  800-22,  CFE  vs  GFE  Selection  Process. 

AFSCR/AFLCR  800-31,  Government-Furni  shed  Ecu  ipmer.t/Contractor- 
Furnished  Equipment  (GFE/CFE)  Selection  Process,  GFE  Acquisi¬ 

tion,  and  GFE  Management. 

SUPPORT  EQUIPMENT  (SE) 

AFR  8-2,  Air  Force  Technical  Order  System. 

AFR  66-14,  Equipment  Maintenance  Policies,  Objectives,  and 
Responsibilities . 

AFR  80-14,  Test  and  Evaluation. 

AFR  800-14,  Transfer  of  Program  Management  Responsibility. 

AFR  800-12,  Acquisition  of  Support  Equipment. 

AFR  800—14,  Vol  I,  Management  of  Computer  Resources  in  Systems. 

AFR  66-1,  Maintenance  Management. 

AFM  67-1,  Vol  I,  Part  1,  Chapter  6;  Vol  III,  Part  7;  Vol  IV,  Part 
1,  Chapter  28  (Table  of  Allowance  System). 

AF LCR  65  —  5  ( Chapter  41),  AF  Provisioning  Policies  and  Procedures. 
AFLCR  66-37,  Management  of  Automated  Test  Systems.. 
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AFLCR  67-2,  USAF  Equipment  Allowance  System. 

AFLCF  67  14,  AF  Equipment  Allowance  Management  Program. 
AFSCR/AFLCR  800-5,  AGE  Acquisition  Management. 

" Copier ' Sy.t« 
AFLCH  57-2,  Comgutatlon  of_Re3ulrgn..nts  for  Equipment  Tvn-  Items 

^gf^f^ulsltlon,  ana 

ot  Avionles 

MIL-STD-1519 ,  Test  Requirements  Documents. 

MIanfg;“^;r^4^-^Vle"S  ■°a-asaitLi«.S,yt«.,  Equipment. 

Synthesis!  *  £aft>  Fault_Diagnosis ,  Subsystems,  Analysis/ 

MIL-1- 45208 ,  Inspection  Requirements. 

MIL— Q-9858 ,  Quality  Program  Requirement g . 

MIL-HDBK-300  USAF,  Technical  Information  File  of  SE 
TO  00-20-4,  Configuration  Management  Systems  - 
TO  00-35D-54,  Materiel  Deficiency  Reporting  System. 

AFR  800-22,  CFE  vs  GFE  Selection  Process. 

AFSCR/AFLCR  800-31,  CFE  vs  GFE  Selection  Process. 

MANAGING  CONTRACTOR  DATA  ACQUISITION 

Sec  I,  Definition  of  Data  and  t  v.  i  i  .  , 

Sec  III,  Estimated  Data  Prices  (DD  Form  1423). 

Sec  IV,  Scientific  and  Techhical  Reports. 

Sec  VII,  Rights  in  Data;  Deferred  Delivery  of  T>0nhn1r.i  . 
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Sec  IX,  Rights  in  Technical  and  Other  Data  and  Copyrights; 
Acguisition  of  Technical  Data. 

Sec  XVI,  Contract  Data  Reguirements  List  (DD  Form  1423); 
Management  Systems  Summary  List  ( DD  Form  1660);  Data  Item 
Description  ( DD  Form  1664). 

Sec  XX,  Uniform  Contract  Line  Item  Numbering  System;  Contract 
Exhibits ;  Exhibits  Line  and  Subline  Items. 

DD  Form  250,  Material  Inspection  and  Receiving  Report. 

AFR  310-1,  Management  of  Contractor  Data. 

AFR  310-3,  AFSC/AFLC  Sup  1,  Acguisition  and  Management  of  Data 
for  Follow-on  Procurements. 

AFSCR  310-1,  Management  of  Contractor  Data. 

AFLCR  310-1,  Acquisition  Management  of  Contractor  Data. 

AFSC/AF LCR  310-2,  Deferred  Requisitioning  of  Engineering  Data. 

DODD  5000. 19L,  Vol  II.  Acquisition  Management  Systems  and 
Data  Requirements  Control  List. 

ENGINEERING  DATA 

AFR  57-6/AFLC  Supplements  1  and  2,  DOD  High  Dollar  Spare  Parts 
Breakout  Program. 

AFSCR  310-1,  Management  of  Contractor  Data. 

DOD-D-1000 ,  Drawings,  Engineering  &  Associated  Lists. 
DOD-STD-100,  Engineering  Drawing  Practices. 

MIL-STD-789,  Procurement  Method  Codinq  of  Replenishment  Spare 
Parts.  "T  - - c - 

MIL -STD-885,  Procurement  Data  Packages. 

CONFIGURATION  MANAGEMENT  (CM) 

MIL-S-83490,  Specifications  Types,  and  Forms. 

MIL-STD-490,  Specifications  Practices. 

DOD-STD-480A,  Configuration  Control-Engineering  Changes, 
Deviations,  and  Waivers. 


MIL-STD-481A,  Configuration  Control-Engineering  Changes, 
Deviations  and  Waivers  (short  Form) . 

MIL-STD-482A,  Configuration  Status  Accounting  Data  Elements, 
and  Related  Features. 

MIL-STD-483 (USAF ) ,  Configuration  Management  Practices  for 
Systems,  Equipment,  Munitions,  and  Computer  Programs. 

MIL-STD-1521A,  Technical  Reviews  and  Audits  for  Systems, 
Equipments,  and  Computer  Programs. 

AFR  800-4 ,  Transfer  of  Program  Management  Responsibility. 

AFR  800—14,  Vol  II,  Acquisition  and  Support  Procedures  for 
Computer  Resources  in  Systems. 

AFR  65-3/AFSC  Sup  1,  Configuration  Management. 

AFSCP  800-7,  Configuration  Management. 

TO  00-20-4,  Configuration  Management  Systems. 

DOD-STD-100C ,  Engineering  Drawing  Practices. 

MIL-STD-130 ,  Identification  and  Marking  of  US  Military  Property. 

DOD-D-1000B,  Drawing,  Engineering  and  Associated  Lists. 

MIL— STD— 196 ,  Joint  Electronic  Typed  Designation  System. 

MIL-STD-155,  Joint  Photographic  Type  Designation  System. 

MIL-STD-875 ,  Type  Designation  System  for  Aeronautical  and  Support 
Equ ipment . 

PROGAM  MANAGEMENT  RESPONSIBILITY  TRANSFER  ( PMRT) /SYSTEM  OR 

EQUIPMENT  TURNOVER 

AFR  800-4 /AFLC-AFSC  Sup  1,  Transfer  of  Program  Management 
Responsibility 

AFR  800-19/AFLC  Sup  1,  System  or  Equipment  Turnover. 

AFSCP  800-3,  A  Guide  for  Program  Management. 

AFLCR  400-1,  Logistics  Management  Policy. 

AFLCR  800-9,  Management  of  DPML/ILSOs. 

METROLOGY  AND  CALIBRATION 

AFR  74-2,  and  AFLC  Sup  1,  Air  Force  Metrology  and  Calibration 
Program. 
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AFR  400-3/AFLC  Sup  1,  Foreign  Military  Sales. 

TO  00-20-14 ,  Air  Force  Metrology  and  Calibration  Program. 

TECHNICAL  ORDERS  (TO) 

AFR  8-2,  Air  Force  Technical  Order  Systems, 

AFR  310-1,  Management  of  Contractor  Data. 

AFR  400-3,  Foreign  Military  Sales. 

AFM  67-1,  Vol  IX,  USAF  Supply  Manual. 

AFSCM  310-2,  Technical  Publications  Acquisition  Management. 

AFAD  71—531,  Series,  Technical  Order  Data  Reguirements . 

> 

TO  00-5-1,  AF  Technical  Order  System. 

TO  00-5-2,  Technical  Order  Distribution  System. 

TO  00-5-15,  AF  Time  Compliance  Technical  Order  System, 

MIL-M-7298 ,  Manuals,  Technical:  Commercial  Equipment. 

MIL-N-7384 ,  Notices,  Contractor  Furnished  Equipment. 

MIL-L-8031 ,  List  of  Applicable  Publications  (LOAPs). 

MIL-M-24 100 ,  Manuals,  Technical:  Functionally  Oriented  Maintenance 
Manuals .  . 

MIL-M-83495 ,  Manuals,  Technical,  Organizational  Maintenance 

Manual  Set:  General  Requirements  for  Preparation  of  ( for  Air¬ 

craft  Missiles  and  Space  Vehicles.) 

HQ  AFLC/LOLDT,  Specification  List  Exhibit. 

COMPUTER  RESOURCES 

AFR  26-1,  Manpower  Policies  and  Procedures. 

/  t 

AFR  50-9,  Special  Training. 

I 

AFR  57-1,  Statement  of  Operational  Need  (SON). 


AFR  57-4,  Modification  Prog 

cam  Approval. 

AFR  65-3,  Configuration  Man 

ngement . 

AFR  66-12,  Aircraft  and  Miti 

»ile  Equipment  Accountability 

4 


♦ 


Maintenance  Objective,  and 

^evelo^ar  °f  A"  F°rCe  TW»°"«  ^Siearch  and 
AFR  80 - 14 ,  Test  and  Evaluation . 

AFR  86-1,  Programming  Civil  Engineer  Resources. 

AFR  100-18,  USAF  Ground  Communications  -  Electronic  Plannina  and 
Program  Management.  - - “ - -  -  Planning  and 

AFM  172-1,  USAF  Budget  Manual. 

^Process  inqLprogram!  °f  «1£  USAF  Automata  date 

f°r  Man^lnc>  ^225tea  Data 

AFR  310-1,  Management  of  Contractor  Data. 

AFR  800-2,  Acquisition  Program  Management. 

AFR  800-3,  Engineering  of  Defense  Systems. 

A^R  800-4,  Transfer  of  Program  Management  Responsibilities. 

AFR  800_’2/  Acquisition  of  Support  Equipment. 

AFR  800-14,  Vol  I,  Management  of  Computer  Resources  in  Systems. 
AFR  800-19,  System  or  Equipment  Turnover. 

AFLCR  80-3,  Engineering  Planning  and  Programming. 

AFLSR  80-14,  Test  and  Evaluation, 

^pSL800'21/,  MgqagCTient  and  Support  Procedures  for  Computer 
Resources  Used  in  Defense  Systems.-  ‘  ~ — - - — ° - 

AFSCR  27-1,  Program  Direction. 

AFTECR  55-1,  AFTEC  Operations  Regulation. 

MIRelated8FeaEJ£es?Urai:^0n  StatU-3-  ^°°untin?  Elements  and 

MIL-STD-483 ,  Configuration  Management  Practices  for  Systems 
Equipment,  Motions,  and  Computer'~Programs“ - ' 

MIL-STD-490,  System  Specifications. 

MIL-STD-499,  System  Engineering. 
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Revie“s  a"d  ^its  te  Equ iproent . 

DOD-STD-lOO 


PACKAGING,  HANDLING,  AND  TRANSPORTATION 
DODI  5105. 38M,  The  Military  Assistance  and  Sales  Manual 
AFR  71-1'  Packaging  Management-. 

AFR  80-18,  POD  Engineering  for  Transpor  tahi  1  -i  t-y 
AFM  75-2,  Military  Traffic  Management  Regulation. 

AFR  76"18'  —  TransP°rtabilitv  Test  Loading  Agency 
^Procedures—^  PaCkaq-in9  and  Materials  Handling  Policies  and 

AFSC  DH  1-11,  Air  Transportability. 

AFC DM  84-1,  Manufacturing  Operation. 

MIL-STD-1388-1A,  Logistic  Support  Analyse: 

AFR  75-1,  Transportation  of  Materiel. 

AFMater ie  1 . ^-ansPortat °1  Foreign  Military  Sales  (FMSj 


AFLCR/AFSCR  800-29.  ParVaninn  ^ 

Hazardous  Materials^  ^ ^ - £.ansPortation  Certification  of 


QUALITY  ASSURANCE  (QA) 

~0!auses-qsectio°nx^q^at1'On  ( nAP  1  Secti°"  VII,  Contract 
Ajji-ndtx  X, 

Foreign  wil°taryasalesPSh?nments.0f  PiscrePancY  Sports  Against 
AFR  74-1,  Quality  Assurance  Program. 

°f  Pr°d“Ct  ^alitx  Deficiency 
TO  °0-5-1'  AF  Technical  Order  Program. 

T°invest  igat  inn '  SygyCm^ater  16 1  De.ficiency_  Report  inq  and 
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AFLCR  66  15 ,  Product.  Performance  « 

AFSupportCEauipmt,At.~~t;enanCe  °f  Aer°Space  Vehicles  and  Related 
AFSCR  74-1,  Quality  Assurance  Program. 

AFSCR  74-2,  Quality  Assurance  Program  for  Centers  and  Ranges. 
AFSCR  84-2,  Production  Readiness  Review* ** 

^  Pi«PQ»ition  System  for 

Aud^s  for  Sys^ns^u^- 
MIL-STD-1528  (USAF),  Production  Management. 

MIL-STD-1535A,  Supplier  Quality  Assurance  Program  Requirements. 
MIL-Q-9858A,  Quality  Program  Requirements. 

i~45208A,  Inspection  System  Requirements. 
v'Hj-C--45662A,  Calibration  System  Requirements. 

MIL-S-52779  (AD),  Software  Quality  Assurance  Program  Requirements 


TEST  AND  EVALUATION  (T&E) 

AFR  23-36,  Air  Force  Test  and  Evaluation  Center  (AFTEC). 

AFM  55-43,  Vols  I  and  II,  Management  of  Operational  Test  and 
Evaluation «  ~~  ~  ~  "  ■  1  ~  •***  — — — — — -  ■  ■  . 

AFR  80-5 '  Air  Force  Reliability  and  Maintainability  Program. 
AFR  80-14  and  its  AFSC  supplement,  Test  and  Evaluation. 

AFLCR  809—4  ,  Test  and  Evaluation ♦ 

**  * 

AFTECR  55-1,  AFTEC  Operations. 

AFTECP  400—1,  Logistics  Assessments. 


CLASS  V  MODIFICATIONS 
AFR  65-3,  Configuration  Management. 

AFR  57—4,  Modification  Program  Approval. 

AFR  400-3,  Foreign  Military  Sales. 
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LESSONS  LEARNED 


AFR  66-30,  Product  Improvement  Program  for  Operational  Equipment. 

AFR  178-7,  Management  and  Control  of  Information  Requirements. 

AFLCR/AFSCR  400-42,  International  Logistics  Foreign  Military 
Sales  Lessons  Learned. 

AFSCR  800—10,  AFSC  Lessons  Learned  Program. 

INTERNATIONAL  LOGISTICS  (IL) 

AFR  400—3,  Foreign  Military  Sales  and  AFLC  Supplement  1. 

AFR  400-19,  Security  Assistance  Program  (SAP)  Ground  Communications. 

AFM  67-1,  Vol  IX,  Security  Assistance  Program  Procedures. 

AFR  73-3,  Standardization  and  Interoperability  of  Weapon  Systems 
and  Equipment  in  the  North  Atlantic  Treaty  Organization  (NATO)'. 

TRAINING  OF  ILSO  PERSONNEL 
AFR  40-411,  Employee  Training  and  Developmr.t . 

AFM  50-5,  Vol  I  and  II,  USAF  Formal  Schools  Catalog. 

DOD  Catalog  5010.16-C,  Defense  Management  Education  and  Training. 

AFR  50-9,  Special  Training. 

STANDARD  INTEGRATED  SUPPORT  MANAGEMENT  SYSTEM  (SISMS) 

AFR  800-8,  Integrated  Logistics  Support  (ILS)  Program. 

AFLCR/AFSCR  800-24,  Standard  Integrated  Support  Management  System. 

AFSCR/AFLCR  800-2,  Management  of  Multiservice  Systems,  Programs 
and  Projects  “  2 - 

AFR  800-10,  Management  of  Multiservice  and  Agency  Systems, 

Programs,  and  Projects. 

MANUFACTURING  MANAGEMENT 
AFSCM  84-3,  Production  Management. 

AFR  78-3,  Manufacturing  Technology  Program. 
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AFR  800  9,  Manufacturing  Management  for  Air  Force  Acquisition. 
MIL-STD-1528,  Production  Management. 

25  /  Production  Surveillance  and  Reporting. 


^  ■  11  i  “i  “  j  -  "  t  -  -  w  -  v  i  ■  i  ^  /|r  ^  ^ ^  ux  vjn  r 

Acquisition,  and  GFE  Management ~ 

PREOPERATIONAL  SUPPORT 


DA7-104!l00)?eCti°n  VI1  (includin5  AFSC  Supplement  and  especially 


AFR  67-19  (including  AFSC  Supplement)  Logistic  Support  of 
Research,  Development,  Test  and  Evaluation  Activities.  ' 

AFM  172-1,  Vol  I,  Chapter  14,  USAF  Budget  Manual. 

AFR  400-26,  Logistics  Support  for  Ground  Communications  - 
Electronics  (C-E)  Systems  and'  Equipments.  “  “ 

AFSCR/AFLCR  66-24,  Maintenance  of  Aerospace  Vehicles  and  Related 
Support  Equipment.  “  - - - - - - - — - 

AFLCR/AFSCR  800-24,  Standard  Integrated  Support  Management  System. 

AFSCR  66-5,  System  Effectiveness  Data  System  Recording  Procedures. 

AFSCR  67-6,  Authorization  and  Control  of  RDT&E  Equipment  Allowance 
Source  C odes  ( ASC )  04 0  and  04 9  .  -  — c - - 

AFSCR  67-7.  Excess  Experimental  and  Non-NSN  RDT&E  Equipment. 

AFSCM  75-1,  Systems  and  Procurement  Transportation  (Chapter  7). 

^k-STD-1538  (USAF),  Spare  Parts  and  Maintenance  Support  of 
Space  and  Missile  System  Undergoing  RDT&E . 

TO-OO-5-2 ,  Technical  Order  Distribution  System. 

TO-00-25-107 ,  Maintenance  Assistance. 


AFR  800-8,  Inte 
supplemented . 


NETWORK  ANALYSIS  (NA) 

*  Integrated  Logistics  Support  (ILS)  Proqram,  as 

orl  r  *  - - -  —  .  --■■■* 


A- 16 


SELECTED  RISK  ANALYSIS  REFERENCES 
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.  ■  >  r 

Arthur  D.  Little,  Inc.,  Research  Needs  for  Dealing  with  Uncer¬ 
tainty  in  Risk  Analysis,  PB84-150127,  September  1983. 
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APPENDIX  B.  GLOSSARY  OF  TERMS  AND  ABBREVIATIONS 


This  appendix  contains  two  parts.  The  first  is  a  glossary 
of  terms  frequently  used  in  acquisition  and  logistics  (pages  B-l 
to  B-ll).  These  terms  are  briefly  defined  with  a  pertinent 
source  of  additional  information  noted  after  the  definition. 
Following  these  definitions  is  a  list  of  acronyms  used  in  acqui¬ 
sition  and  logistics  (pages  B-12  to  B-16). 


GLOSSARY  OF  TERMS 


Air  Force-Designated  Acquisition  Program  (AFDAP) .  A  system 
acquisition  program  not  qualifying  as  a  major  system  acquisition, 
^ut  determined  by  the  SAF  to  be  of  such  importance  and  priority 
that  it  requires  special  management  attention  and  Secretarial- 
level  milestone  decisions.  (AFR  80-14) 

Air  Force  Systems  Acquisition  Review  Council  (AFSARC).  The 
senior  Air  Force  advisory  council  for  system  acquisition.  Its 
membership  includes  the  Under  Secretary  of  the  Air  Force,  the 
Assistant  Secretaries,  the  Vice  Chief  of  Staff,  and  designated 
Deputy  Chiefs  of  Staff.  The  AFSARC  reviews:  major  programs  as 
part  of  the  SECDEF  milestone  decision  process;  other  designated 
acquisition  programs  before  decisions  by  the  SAF;  and,  in  special 
instances ,  other  acquisition  programs  when  the  decision  to  be 
made  is  of  such  importance  that  is  requires  SAF  attention.  (AFR 


Automatic  Test  Equipment  (ATE ) .  ATE  is  a  generic  term  for  equip- 
ment  (separate  or  built  in)  satisfying  a  test  function  (diagnos- 
tic  or  condition  indicating)  and  possessing  an  automatic  capa¬ 
bility.  In  this  sense,  ATE  can  be  either  a  part  of  the  mission 
equipment  or  it  can  be  a  part  of  support  equipment.  (Also,  see 
MIL-STD-1309 . )  (AFR  800-12) 

Price.  A  negotiated  amount  that  specifies  the  maximum 
liability  of  the  Government  for  a  given  acquisition. 

Change  Order .  A  written  order  signed  by  the  contracting  officer, 
directing  the  contractor  to  make  changes  that  the  Changes  clause 
of  the  contact  authorizes  the  contracting  office  to  make  without 
the  consent  of  the  contractor.  (Defense  Acquisition  Regulation 
Manual  (DARM)  No.  1) 

Competitive  Negotiation.  A  negotiated  acquisition  that  (1)  is 

by  a  Request  for  Proposals,  which  sets  out  the  Govern¬ 
ment’s  requirements  and  the  criteria  for  evaluation  of  offers, 

(2)  contemplates  the  submission  of  timely  proposals  by  the 
maximum  number  of  possible  offerors,  (3)  usually  provides 
discussion  with  those  offerors  found  to  be  within  the  competitive 
range,  and  (4)  concludes  with  the  award  of  a  contract  to  the  one 
offeror  whose  offer,  price  and  other  factors  considered,  is  most 
advantageous  to  the  Government.  (DARM  No.  1) 

Computer  Program.  A  series  of  instructions  of  statements  in  a 
form  acceptable  to  an  electronic  computer,  designe#d  to  cause  the 
computer  to  execute  an  operation  or  operations.  (*AFR  800-14. 
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Computer  Program  Identification  Number  (CPIN).  A  variahie 

StrSrer^  nri£l6r  US6d  to"P^v'^e  unique  identification, 

fi'.  and  management  of  Embedded  Computer  Systems  (ECS)  soft¬ 
ware  in  the  Air  Force.  ( AFLCR  800-21) 

Computer  Resources.  The  totality  of  computer  equipment ,  computer 

nf?g™3'  associated  documentation,  contractual  services ,  person¬ 
nel  and  supplies.  (AFR  800-14,  vol .  I)  person 

^IfernativS1^?1^11  Phase’  Th?  identification  and  exploration  of 
alternative  solutions  or  solution  concepts  to  satisfy  a  validated 

industr^and*  Jhro^?h  .  use  of  contracts  with  competent 
industry  and  educational  institutions.  This  phase  requires  the 

?aid!l  involvement  of  all  participating  commands  to  identify  the 
S2diSte^-Sa1Ut1^^  and  thelr  characteristics.  One  or  more^of 
•  ®elected  candidate  solutions  are  then  approved  for  entry  into 
the  Demonstration  and  Validation  phase.  y 

£°g.flguration  (Change)  Control  Board  (CCB).  A  board  composed  of 
representatives  from  program/project  functional  areas  subh  as 

conf iguration  management,  contracting,  manufactur- 
i  g,  test  and  logistic  support,  training  activities  and  usinq/ 
supporting  organizations.  This  board  approves  or  disapproves 
proposed  change  requests.  (AFSCP  800-7) 

£2n-^.:*-l2Hrat:‘-crj _ Item/ Computer  Program  Configuration  Item  (Cl/CPCl) 

An  aggregation  of  hardware/ computer  ‘progrlms  or 'any  of!  its  ‘ 

discrete  portions,  which  satisfies  an  end-use  function  and  is 
designated  by  the  Government  for  configuration  management.  (AFR 

Configuration  Management  (CM).  A  discipline  applying  technical 
and  administrative  direction  and  surveillance  to  U ^identify  and 
document  the  functions!  and  physical  characteristics  of  a  con- 

:iaUf?l10n  lt*”'  <2)  oontro1  changes  to  those  characteristics, 
.A  ,  record  and  report  change  processing  and  implementation 
status.  It  includes  configuration  identification,  control, 
status  accounting  and  audits.  (AFSCP  800-7) 

gonstructive  Change.  During  contract  performance,  an  oral  or 
ritten  act  or  omrrussion  by  the  contracting  officer  or  other 
authorized  Government  official,  which  is  of  such  a  nature  that  it 

(SaSHo!"?  t0  haVS  the  Same  e£fe=t  aS  3  «ritten  change  order. 

I^Fh-or?LaaiaaRe<]alrement3  Llst-  A  listln<3  of  data  requirements 
..o  -honzed  and  made  a  part  of  the  contract  on  DD  Form  1423, 

(AFSCM/AFLCM^lO— 1  )lirementS  LiSt'"  ”  "‘e0haniCal  ^-alent. 

Tur-nished*^  Property,  other  than  Government 

t«ct !  (AfsS  2^1)  C°ntraCtor  in  the  Performance  of  a  con- 
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Decision  Coordinating  Paper  (DCP).  The  principal  document  to 
record  essential  system  program  information  for  use  in  support  of 
the  Secretary  of  Defense  decision-making  process  at  Milestones, 

I,  II  and  III.  (DODD  5000.2) 

Defense^Acquisition  Executive.  The  principal  advisor  and  staff 
assistant  to  the  Secretary  of  Defense  and  the  focal  point  in  OSD 
for  system  acquisitions.  (DODD  5000.1) 

Rgf.gnsg  Acquisition  Regulation.  Uniform  policies  for  the  Depart¬ 
ment  of  Defense  relating  to  the  acquisition  of  supplies  and 
services  under  the  authority  of  Title  10,  United  States  Code, 
Chapter  137.  Formerly  called  the  Armed  Services  Procurement 
Regulation.  (AR  320-5) 

Pgfgnse  Systems  Acquisition  Review  Council  (DSARC).  An  advisory 
council  established  by  and  functioning  for  the  Secretary  of 
Defense  ( SECDEF )  to  apprise  the  SECDEF  of  the  program  status  and 
readiness  of  a  major  defense  system  prior  to  proceeding  to  the 
next  phase  in  the  acquisition  process.  (AFR  80-14) 

itized  Agre ement .  A  contract  that  has  been  signed  by  both 
the  contractor  and  the  Government.  The  term  usually  implies  that 
a  price  has  been  negotiated  and  has  been  reflected  in  the 
contract. 

Demostration  and  Validation  Phase.  The  p ^riod  when  selected 
candidate  solutions  are  refined  through  extensive  study  and 
analyses;  hardware  development,  if  appropriate;  tests;  and 
evaluations.  The  objective  is  to  validate  one  or  more  of  the 
selected  solutions  and  give  a  basis  for  deciding  whether  to 
proceed  into  Full-Scale  Development.  (AFR  800-2) 

PgR^y  Program  Manager  of  Logistics  ( DPML )  .  The  logistics 
representative  for  major  programs  at  the  Program  Office.  He  is 
directly  responsible  to  the  PM  for  all  logistic  tasks.  He 
ensures  that  logistic  participation  and  support  capabilities 
agree  with  program  objectives  and  that  logistics  support  require¬ 
ments  are  reflected  in  the  system  design.  (AFSCP  800-3) 

^gt-friPj-nat^on  and  Finding.  A  formal  document  that  records  the 
decision  of  a  contracting  officer  and  higher  approval,  if  neces- 
sary*  .  Determination  and  Findings  are  used  to  secure  approval  to 
negotiate  rather  than  advertise,  to  use  a  cost  type  contract,  and 
to  exercise  an  option. 

Development  Testing  and  Evaluation  ( DT&E ) .  That  testing  and 
evaluation  used  to  measure  progress,  verify  accomplishment  of 
development  objectives,  and  to  determine  if  theories,  techniques, 
and  materiel  are  practicable;  and  if  systems  or  items  under 
development  are  technically  sound,  reliable,  safe,  and  satisfy 
specifications.  ( AFM  11-1) 
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Follow-on  Operational  Test  and  Evaluation  (FOT&E).  The  second 
p  ase  of  operational  test  and  evaluation  conducted  on  production 
items  in  an  operational  environment.  It  is  conducted  to  verify 
operational  effectiveness  and  operational  suitability  and  to 
provide  information  on  organization,  personnel  requirements, 

and  tactics,  and  assess  whether  production  techniques 
have  affected  system  performance  or  operational  suitability  as 
determined  during  initial  operational  test  and  evaluation.  (AFM 


Irlf  5r  '55  ^  ^  Q?f  ?f  *he  ™a;|0r  methods  of  acquisition 

preferred  by  law  when  it  is  feasible  and  practicable  to  employ 

it.  It  is  used  in  one  of  two  forms,  as  appropriate  to  the  * 

555515^m^nt 1  conventional  formal  advertising  and  two-step  formal 
advertising.  The  first  involves  the  acquisition  of  well-defined 
items  requiring  the  submission  of  technical  proposals  prior  to 

pri°es-  In  b°th  £orms-  is  mad5  £o  thj 

responsible  bidder  whose  bid,  conforming  to  the  invitation  for 
bids,  will  be  most  advantageous  to  the  Government,  price  nd  other 
factors  considered.  (DARM  No.  1) 

lBi:j:-.Scale_  Development  Phase.  The  period  when  the  system  and  the 
principal  items  necessary  for  its  support  are  designed,  fabri- 
cated,  tested,  and  evaluated.  The  intended  output  is,  as  a 
minimum:  a  preproduction  system  that  closely  approximates  the 

inal  product;  the  documentation  needed  to  enter  the  production 
phase;  and  the  test  results  that  show  the  product  will  meet  the 
requirements.  This  phase  includes  the  acquisition  of  long  lead 
production  items  and  limited  production  for  OT&E.  (AFR  800-2) 

Government  Furnished  Equipment.  Separable  equipment  and  compon¬ 
ents  of  a  total  system  acquired  by  the  Navy  and  supplied  to  the 
P-4215 )PrimS  COntractor  for  integrating  into  the  system.  ( NAVMAT 


Government  Furnished  Property.  Property  in  the  possession  of  or 
acquired  by  the  Government  and  delivered  or  otherwise  made 

?AFSCM^57 - 2°  a  contractor  for  use  in  accomplishing  a  contract. 


H^ad  of  Acquiring  Activity.  The  chief,  commander,  or  other 
orticiai  in  charge  of  an  acquiring  activity.  Acquiring 
activities  include  such  commands  as  Air  Force  Systems  Command, 
^DAR^°rCe  Logistlcs  Command,  Tactical  Air  Command,  and  others. 


Implementing  Command.  The  command  (normally  AFSC)  charged  with 
responsibility  for  acquiring  systems  and  equipment  for  the  Air 
Force  inventory.  (AFR  800-4) 
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Initial  Operational  Test  and  Evaluation  (IOT&E).  The  first  phase 
of  operational  test  and  evaluation  conducted  on  preproduction 
items,  prototypes,  or  pilot  production  items  and  normally 
completed  prior  to  the  first  major  production  decision.  It  is 
conducted  to  provide  a  valid  estimate  of  a  system’s  operational 
effectiveness  and  operational  suitability  prior  to  the  first 
major  production  decision.  ( AFM  11-1) 

Integrated  Logistics  Support  (ILS).  A  unified  and  iterative 
approach  to  the  management  and  technical  activities  necessary  to: 

(1)  cause  support  considerations  to  influence  both  requirements 
and  design;  (2)  define  support  requirements  that  are  optimally 
related  to  the  design  and  to  each  other;  (3)  acquire  the  required 
support;  and  (4)  provide  for  the  required  support  in  the 
operational  phase  at  minimum  cost.  (AFR  800-8) 

Integrated  Logistics  Support  Manager  (ILSM).  An  experienced 
logistician  who  is  assigned  to  manage  ILS  for  programs  not 
designated  as  major  programs.  (AFR  800-8) 

Integrated  Logistics  Support  Plan  (ILSP).  An  Air  Force  manage¬ 
ment  plan  developed  and  used  by  the  program  manager  and  the  DP ML 
or  ILSM,  to  manage  the  ILS  process.  This  includes  the  horizontal 
integration  of  the  ILS  elements  (that  is,  with  each  other),  as 
well  as  their  vertical  integration  into  the  various  aspects  of 
program  planning,  engineering,  designing,  testing,  evaluating, 
and  during  production  and  operation.  It  uiso  includes  the  inte¬ 
gration  of  support  elements  with  the  mission  elements  of  a  system 
throughout  its  life  cycle,  and  is  updated  as  the  program  evolves. 

The  ILSP  is  a  part  of  the  Program  Management  Plan  (PMP)  and,  when 
approved,  becomes  directive  on  all  participating  agencies.  (AFR  800-8) 

Invitation  for  Bids.  The  soliciation  document  used  in  conven¬ 
tional  formal  advertising  and  in  the  second  step  of  two-step 
formal  advertising.  (DARM  No.  1) 

Item  Manager  (IM).  The  AFLC  ALC  (or  other  service  or  agency) 
assigned  the  management  responsibility  for  commodity-type  items 
by  Federal  Supply  Class.  (AFSCP  800-7) 

Learning  Curve.  A  tool  of  calculation  used  primarily  to  project 
resource  requirements,  in  terms  of  direct  manufacturing  labor 
hours  or  the  quantity  of  material  (for  this  purpose,  usually 
referred  to  as  an  improvement  curve)  required  for  a  production 
run.  Used  interchangeably  with  the  term  "improvement  curve,"  the 
concept  of  a  learner’s  curve  was  adopted  from  the  observation 
that  individuals  who  perform  repetitive  tasks  exhibit  a  rate 
of  improvement  due  to  increased  manual  dexterity.  Learning  or 
improvement  curve  theories  include  the  following; 

•  The  Boeing  or  unit  curve  theory:  As  the  total  quantity 
of  units  produced  doubles,  the  cost  per  unit  decreases 
by  some  constant  percentage  (the  rate  of  learning). 
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•  The  Northrup  or  cumulative  average  theory:  As  the 

total  quantity  of  units  doubles,  the  average  cost  per 
unit  decreases  by  some  constant  percentage  (the  rate  of 
learning).  (DARM  No.  1) 

Letter — of  Of fer /Acceptance  ( LDA ) .  A  Document,  KDD  1513,  that 
records  the  offer  of  the  United  States  to  sell  and  the  foreign 
government's  agreement  to  buy  a  given  article  or  service. 

Cycle  Cost  (LCC).  The  total  cost  of  an  item  or  system  over 
its  full  life.  It  includes  the  cost  of  acquisition,  ownership 
(operation,  maintenance,  support,  etc.)  and,  where  applicable, 
disposal.  To  be  meaningful,  an  expression  of  life  cycle  cost 
must  be  placed  in  context  with  the  cost  elements  included,  period 
of  time  covered,  assumptions  and  conditions  applied,  and  whether 
it  is  intended  as  a  relative  comparison  or  absolute  expression  of 
expected  cost  effects.  (AFR  800-11) 


— _ System  Acquisition.  A  system  acquisition  program  desig¬ 

nated  by  the  Secretary  of  Defense  to  be  of  such  importance  and 
priority  as  to  require  special  management  attention.  (DODD 
5000.1) 


Milestone  0  Decision.  Approval  of  MENS  and  authorization  to 
proceed  into  Phase  0 — Concept  Exploration — which  includes  solici¬ 
tation,  evaluation  and  competitive  exploration  of  alternative 
system  concepts.  Approval  to  proceed  with  Concept  Exploration 
also  means  that  the  Secretary  of  Defense  intends  to  satisfy  the 
need.  (DODD  5000.1  ) 


Milestone  I  Decision.  Selection  of  alternatives  and  authoriza¬ 
tion  to  proceed  into  Phase  I — Demonstration  and  Validation. 

(DODD  5000.1  ) 

Milestone  II  Decision.  Selection  of  alternati ve ( s )  and  authori¬ 
zation  to  proceed  into  Phase  II-Full-Scale  Development— which 
includes  limited  production  for  operational  test  and  evaluation. 
Approval  to  proceed  with  Full-Scale  Development  also  means  that 
the  Secretary  of  Defense  intends  to  deploy  the  system.  (DODD 


Milestone  III  Decision.  Authorization  to  proceed  into  Phase 
III  —  Production  and  Deployment.  (DODD  5000.1) 

Mission  Area  Analysis.  Continuous  analysis  of  assigned  mission 
responsibilities  in  several  mission  areas,  to  identify  deficien¬ 
cies  in  the  current  and  projected  capabilities,  to  meet  essential 
needs,  and  to  identify  opportunities  for  the  enhancement  of  capa¬ 
bility  through  more  effective  systems  and  less  costly  methods. 
(AFR  57-1)  * 
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Mission  Element  Need  Statement  (MENS).  A  statement  prepared  by 
DOD  Component  to  identify  and  support  the  need  for  a  new  or 
improved  mission  capability.  The  mission  need  may  be  the  result 
of  a  projected  deficiency  or  obsolesence  in  existing  systems,  a 
technological  opportunity,  or  an  opportunity  to  reduce  operating 
cost.  The  MENS  is  submitted  to  the  Secretary  of  Defense  for  a 
Milestone  0  decision.  (DODD  5000.2) 

Not-To-Exceed  Price.  An  amount  stipulated  by  the  contractor  for 
which  he  will  do  a  defined  amount  of  work.  Not-to-exceed  prices 
are  typically  submitted  as  part  of  Class  I  engineering  change 
proposals.  If  the  contracting  officer  decides  to  order  the 
contractor  to  make  the  change  by  means  of  a  change  order,  the 
not- to— exceed  price  is  stipulated  in  the  change  order  and 
represents  the  maximum  amount  the  contractor  can  collect  for 
performing  the  work. 

Operational  Test  and  Evaluation  (OT&E).  Testing  and  evaluation 
(divided  into  initial  operational  test  and  evaluation  and 
follow-on  operational  environment  as  possible  to  estimate  the 
prospective  systems  military  utility,  operational  effectiveness 
and  operational  suitability.  (AFM  11-1) 

Participating  Command.  A  command  or  agency  designated  by  HQ  USAF 
to  support  and  advise  the  PM.  A  supporting  command  is  also  a 
participating  command.  (AFR  800-2) 

Price  Negotiation  Memorandum.  The  document  that  relates  the 
story  of  the  negotiation.  It  is  first  a  sales  document  that 
establishes  the  reasonableness  of  the  agreement  reached  with  the 
successful  offeror.  Second,  it  is  the  permanent  record  of  the 
decisions  the  negotiator  made  in  establishing  that  the  price  was 
fair  and  reasonable.  Called  the  PNM.  (DARM  No.  1) 

Product  Division.  Those  organizational  bodies  within  AFSC 
responsible  for  systems  acquisition.  These  bodies  are  ASD,  ESD, 
AD,  BMD,  and  SD. 

Production  and  Deployment  Phase. 

(1)  The  period  from  production  approval  until  the  last 
system  is  delivered  and  accepted.  The  objective  is  to 
efficiently  produce  and  deliver  effective  and  support¬ 
able  systems  to  the  operating  units.  This  includes  the 
production  of  all  principal  and  support  equipment. 

(2)  Deployment.  The  period  encompassing  the  process  of 
uniting  facilities,  hardware  and  software,  personnel, 
and  procedural  publications;  and  delivering  an  accep¬ 
table  integrated  system  to  the  using  and  supporting 
commands.  This  overlaps  the  production  phase.  (AFR 
800-2) 
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Program  Assessment  Review.  Quarterly  status  review  of  each  major 
system  program.  Normally,  a  30  minute  presentation  by  the 
program  manager.  Program  assessment  review  presentations  are 
made  to  the  Commander,  AFSC.  (AFSCP  800-3) 

Program  Management  Directive  ( PMD ) .  The  HQ  USAF  document  that 
directs  the  implementing  and  participating  commands  and  satisfies 
documentation  requirements.  It  is  used  during  the  entire  acqui¬ 
sition  life  cycle  to  state  requirements  and  request  studies,  as 
well  as  initiate,  approve,  transfer,  modify,  or  terminate 
programs.  The  content  of  the  PMD  is  tailored  to  the  needs  of 
each  program.  (AFR  800-2) 

Program  Management  Plan  (PMP).  The  document  developed  by  the  PM, 
with  assistance  from  the  participating  commands.  It  shows  the 
program  objectives  as  well  as  the  integrated  time-phased  activi¬ 
ties  and  resources  required  to  complete  the  task  specified  in  the 
PMD.  The  PMP  is  tailored  to  the  needs  of  each  program,  is 
approved  and  issued  by  the  PM,  and  is  directive  on  all  partici¬ 
pating  commands.  (AFR  800-2) 

Program  Management  Responsibility  Transfer  (PMRT).  The  transfer 
of  program  management  responsibility  for  a  system  (by  series),  or 
equipment  (by  designation),  from  the  implementing  command  to  the 
supporting  command.  PMRT  includes  transfer  of  engineering 
responsibility.  (AFR  800-4) 

Program  Manager  (PM).  The  single  Air  Force  manager  (system 
program  director,  program/project  manager,  or  system/item 
manager)  during  any  specific  phase  of  the  acquisition  life  cycle. 
(AFR  800-2) 

Program  Office  ( PO ) .  The  office  of  the  PM  and  the  single  point 
of  contact  with  industry,  Government  agencies,  and  other  activi¬ 
ties  participating  in  the  system  acquisition  process.  It  is  the 
office  the  program  manager  sets  up  for  the  acquisition  of 
systems,  subsystems,  equipment,  munitions,  or  modifications  to 
them.  (AFR  800-2) 

Progress  Payment.  A  payment  made  as  work  progresses  under  a 
contract  on  the  basis  of  percentage  of  completion  accomplished, 
or  for  work  performed  at  a  particular  stage  of  completion.  (DARM 
No.  1  ) 

Purchase  Request.  An  authenticated  document  prepared  by  a 
purchasing  office  that  stipulates  the  quantities  and  delivery 
dates  of  supplies  or  services.  Purchase  requests  authorize  the 
contracting  officer  to  acquire  the  items. 

Qualification  T&E  (QT&E).  T&E  performed  in  lieu  of  DT&E  on 
programs  for  which  there  is  no  research,  development,  test  and 
evaluation  ( RDT&E )  funding.  (AFR  80-14) 
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Qualification  Ot&f  (oot/cp  ^ 

programs  for  which — re - ~  -  T&E  perform i  ,  . 

evaluation  (RDT&e?  f^f_ls  n°  research,  de^ion^  °f  10T&E  °n 

funding.  ( AFR  80-14)  eveloPment,  test  and 
Request  for  PronQSa, 

t^rned?^  ‘’-tween  the.  Air 

by  conveylng^^^^ntractor  is  int^duclf  t  “  is 

and  to  determine  the  caDah??^tandln9  °f  the  work  t^h"106  desired 
efforts.  rfps  contain3?  b  llty  and  Price  of  ♦-?  k  t  be  Performed 

to  obtain  informa?i^nf^9Ua9e,  terms, ^and  con  Jif  °ntraCt°r  ’  s 

”  from  Prospective  bidders  ^pco  necessary 

^OofDefense  ..  S*  (AF?CM  27-1 ) 


Secretary  p£  Defen  ®  bldders-  (AFSCM  27-1 )  Y 

reaffirms  established  need  1Sh^^°9ranT  goals  Tnd  ?°cuments  each 
exceptions  to  acquisition  ??d  pro9ram  SbjeSti!^' thresholds, 
the  direction  and  guidaSS  P?llCy  (when  approSriSf  (  authorizes 

for  the  next  phase  Of ^acqSiSft- °SD'  0JCS'  andOhe  Don  ?nd  provide£ 
q.  acquisition.  (DOd  5000  it  c°*Ponent 

Share  Ratio  a  *  JUvu.i) 

f°r~  ultimate  costs^hi?  *hich  represents  a  -I  • 
dollar  difference  Kthat  1S  translated  infe  resPonsibilitv 

^0/30  shareOatio  J^nTaV fiSJ?  ln  a"y 

contractor  responsibility ^rV  dollar'is  a 

s  i  f  e  R .  *  1 ' 


'  snare  ratio  means  •an 
contractor'*  >-e^  .ns  30  c 

or  s  responsibility 

g  .  ^  2  *  ~m\Tl  iMO.  1  ) 

pm -established 

;:r  rch  ^ 

Sole  Source  t  Characf  •  (APSCP  800-3) 

avan:b1e'’;erf^etPla“'2p°s--«ting0aneua"d  °nlS'  »°»rce,  regard¬ 
ed  (Sometfmf  anCe  “Pability  for  th2  9“e  and  singularly 
source.  "  )  (DARM  No?dl  }nt-r9bangeably  “"tr-ct 

/  in  single 

i^ntd|'pp^it'^ar,tted  SunP-ort  Manaqom^nf  s 

UCS  coLa A  -anage- 
manage  the  logistic*  '  hat  Prides  a  uniform  J°lnt  iogis- 
tions.  ( AFR  800-8)  SUpport  of  multi -service  svtf  t0  Plan  and 

systems  acquisi- 

Support  Equ  i  ntrtpp  4-  q 

eguipment 

of  the  equipment  required^"  equiPment.  it  does  5„ft.Whlch  is  an 
functions.  (AFR  to  Perform  miss  i  L  °!f- not  delude  anv 


f  the  equipment  req'uTr^1?"  -w  —  ~  . 

functions.  ( AFR  800-HT  P6rf°rm  ^^sion  operaUons"01^6  any 

^££Qr_tinq  Comm^na  Tha 

assumeSprogram  AFLC)  Cha^d  with 

command.  (afr  Soo-!tment  reSponsibilitrf?omtthedide?i9nated  to 

'  tne  implementing 
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contracts  in  the  Commerce  Business  Daily.  y 

!n?!dfPI:^rai”f0bjeCtlves  in  the  acquisition  of  De fense  systems 

deployment1"!  th^Defense^vst  3  mis®ion  . nee<?  through  successful 
(DODD  5000.1)  ~  y  em  or  termination  of  the  program. 

VhS&uhz  t 

system  ££ sS 0 and  a  P-ferred 

System  Manager  ( SM ) .  The  AFLC  focal  point  for  intearatino 
managing  the  functinoal  elements  of  logistics  on  a  ?i^  I  h.cfc 
to  ensure  the  support  of  the  assigned  lylteS?  SurinJIhe  ' 

“SiTiMK’cJSS  orffLlnPr°9ram  transf«'  the  SM  provides  a 
(AFR  800-8)  support  planning  concepts. 

Operational  Concept.  A  formal  document  that  describes  the 
(AFR  gQ^J^05®'  emPloyment'  deployment  and  support  of  a  system. 

IfgragfrglfF  (gLnC^r  •usD°CUlnerltS  P(epa«d  in 

providing  a  record  of  a^onS  fSpSJiS 

replacement  or  installation  of  components)  or  in  accompli sMn^ 

sJfuction ^of9aacieo?°i^  chan3e/alteration  to  the  design  or  con- 
800-7)  °  ltS  ass°ciated  support  equipment.  (AFSCP 


Transfer 

Command 
Implemen 
eng  ineer 

Transfer 

Manager 
meriting, 
scope  of 
program. 


-  „  hat  point  ln  time  when  the  designated  Supporting 

accepts  program  management  responsibilities  fromtt^ 
ting  Command.  This  includes  logistic  support  and  related 
ing  and  acquisition  responsibilities.  (AFR  800-4) 

ll^kinTher?WG  InM  -H  A  9rOUP  established  by  the  Program 
1  '*  The  TWG  includes  representatives  from  the  imple- 

t he PT WG 1 ?! f  d 3  nd  Hther  involved  commands.  The  size  and 
UFR  8oS-4fPen  UPOn  the  Size  and  complexity  of  the 


accepts  " responsibility  f  tlm®  when  the  operating  command  formally 
onpraH™6 spons i bi  1  ity  from  the  Implementing  Command  for  the 
peration  and  maintenance  of  the  system,  equipment,  or  computer 
program  acquired.  (AFR  800-19)  H  P  computer 

(°f  computer  programs).  The  process  of 
wifrh  ?£  ?  bhat  the. computer  program  was  developed  in  accordance 

with  the  stated  specification  and  satisfactorily  performs 
the  mission  environment,  the  function(s)  for  which  it  was 
designed.  (AFR  800-14,  Vol.  I)  C  Was 
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_ •  A  document  signed  by  legal  authority  that  authorizes  a 

person  to  become  a  contracting  officer.  Only  warranted  contract¬ 
ing  officers  can  commit  the  Government. 

Work  Measurement  Program.  A  technique  for  collecting  data  on 
work  hours  and  production  of  work  units  to  determine  the  rela¬ 
tionship  between  work  performed  and  work  hours  expended.  Use 
this  relationship  for  personnel  planning,  scheduling,  manufactur¬ 
ing,  budgeting,  performance  evaluation,  and  cost  control.  Use 
recognized  industrial  engineering  techniques  (time  study, 
standards  data,  work  sampling,  “or 
set  labor  time  standards.  ( AFSCR 


predetermined 
84-7  ) 


time  systems)  to 
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abbreviations  and  acronyms 


AGO 

ACN 

ACWP 

AD 

ADP' 

AEDC 

AFALC 

AFCMD 

AFDAP 

AFFTC 

AFLC 

AFOTEC 

AFPR 

AFR 

AFPRO 

AFSC 

AFSARC 

AGERD 

AIS 

ALC 

AP 

ASARC 

ASD 

ASI 

ATE 

Administrative  Contracting  Officer 

Advance  Change  Notice 

Actual  Cost  of  Work  Performed 

Armament  Division 

Automatic  Data  Processing 

Arnold  Engineering  Dvelopment  Center 

Air  Force  Acquisition  Logistics  Center 

Air  Force  Contract  Management  Division 

Air  Force  Designated  Acquisition  Proqram 

Air  Force  Flight  Test  Center 

Air  Force  Logistics  Command 

Air  Force  Operational  Test  and  Evaluation  Center 
Air  Force  Plant  Representative 

Air  Force  Regulation 

Air  Force  Plant  Representatives  Office 

Air  Force  Systems  Command 

Air  Force  Systems  Acquisition  Review  Council 

AGE  Requirements  Document 

Avionics  Intermediate  Shop 

Air  Logistics  Center 

Acquisition  Plan 

Army  Systems  Acquisition  Review  Council 
Aeronautical  Systems  Division 

Amended  Shipping  Instruction 

Automatic  Test  Equipment 

BA 

BCWP 

BCWS 

BES 

BIT 

BMO 

BOA 

BSP 

Budget  Authorization 

Budgeted  Cost  for  Work  Performed 

Budgeted  Cost  for  Work  Scheduled 

Budget  Estimate  Submission 

Built  in  Test 

Ballistic  Missile  Office 

Basic  Ordering  Agreement 

Business  Strategy  Panel 

CAGEL 

CAS 

CCB 

CC  DR 

CDR 

CDRL 

CETS 

CFE 

CFSR 

CG 

Cl 

CM 

CMSEP 

CPAF 

Consolidated  AGE  List 

Contract  Administration  Services 

Configuration  Control  Board 

Contrct  Cost  Data  Report 

Critical  Design  Review 

Contract  Dated  Requirements  List 

Contractor  Engineering  and  Technical  Services 
Contractor  Furnished  Equipment 

Contractor  Funds  Status  Report 

Consolidated  Guidance 

Configuration  Item 

Configuration  Management 

Contractor  Management  System  Evaluation  Proqram 
Cost- Plus -Award -Fee 
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CPC  I 

CPDP 

CPFF 

CPI  F 

CPIN 

CPR 

CRISP 

CRS 

CRWG 

CSA 

C/SCSC 

CSP 

C/SSR 

Computer  Program  Configuration  Item 

Computer  Program  Development  Plan 

Cost-Plus -Fixed-Fee 

Cost-Plus- Incentive -Fee 

Computer  Program  Identification  Number 

Cost  Performance  Report 

Computer  Resources  Integrated  Support  Plan 

Computer  Resources  Support* 

Computer  Resources  Working  Group 

Configuration  Status  Accounting 

Cost/Schedule  Control  System  Criteria 

Contract  Strategy  Paper 

Cost/Schedule  Status  Report  i 

DAE 

DAR 

DCAA 

DCAS 
DCASMA 
DCASR 
DCAS PRO 

Defense  Acquisition  Executive 

Defense  Acquisition  Regulation 

Defense  Contract  Audit  Agency 

Defense  Contract  Administration  Services 

Defense  Contract  Administration  Services  Management  Area 

Defense  Contract  Administration  Services  Region 

Defense  Contract  Administration  Services  Representa¬ 

tives  Office 

DCP 

DD  250 
D&F 

DLA 

DPML 

DPS 

DR 

DRED 

DSAA 

DSARC 

DT&E 

Decision  Coordinating  Paper 

Air  Force  Accepting/Title  Transfer 

Determination  and  Finding 

Defense  Logistics  Agency 

Deputy  Program  Manager  for  Logistics 

Decision  Package  Sets 

Deficiency  Report 

Deferred  Requisitioning  of  Engineering  Drawings 

Defense  Security  Assistance  Agency 

Defense  Systems  Acquisition  Review  Council 

Development  Test  and  Evaluation 

ESMC 

ECP 

EM 

ESD 

Eastern  Space  and  Missile  Center 

Engineering  Change  Proposal 

Energy  Management 

Electronic  Systems  Division 

FA 

FCA 

FCI 

FFP' 

FMR 

FOT&E 

FPIF 

FPLOE 

FQT 

FSN 

FYDP 

Facilities 

Functional  Configuration  Audit 

Functional  Configuration  Identification  '• 

' Firm-Fixed-Price 

Functional  Management  Review 

Follow-On  Test  and  Evaluation 

Fixed-Price- Incentive-Firm 

Fixed-Price  Level  of  Effort 

Formal  Qualification  Testing 

Federal  Stock  Number 

Five  Year  Defense  Program 
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GAO 

GBL 

GFE 

GFP 

General  Accounting  Office 

Government  Bill  of  Lading 

Government  Furnished  Equipment 

Government  Furnished  Property 

ICWG 

IFB 

ILC 

ILS 

ILSM 

ILSP 

ILS  T&E 
IM 

IOC 

IOT&E 

ISP 

Interface  Control  Working  Group 

Invitation  for  Bid 

International  Logistics  Center 

Integrated  Logistics  Support 

Integrated  Logistics  Support  Manager 

Integrated  Logistics  Support  Plan 

Integrated  Logistics  Support  Test  and  Evaluation 

Item  Manager 

Initial  Operating  Capability 

Initial  Operating  Test  and  Evaluation 

Integrated  Support  Plan 

JMENS 

JOP 

JOR 

JPAM 

JPO 

JSPD 

JTD 

JT&E 

JTF 

Joint  Mission  Element  Need  Statement 

Joint  Operating  Procedure 

Joint  Operational  Requirement 

Joint  Program  Assessment  Memorandum 

Joint  Program  Office 

Joint  Strategic  Planning  Document 

Joint  Test  Director 

Joint  Test  and  Evaluation 

Joint  Task  Force 

LCC 

LOA 

LRU 

LSA 

LSM  I 

LSRF 

Life  Cycle  Cost 

Letter  of  Offer/Acceptance 

Line  Replaceable  Unit 

Logistic  Support  Analysis 

Logistics  Support  Management  Information 

Logistics  Support  Resource  Funds 

MAA 

MACSO 

MDR 

MENS 

MIPR 

MOA 

MPT 

MP 

MSI 

MRP 

MST&E 

MTS 

Mission  Area  Analysis 

Military  Airlift  Command  Support  Office 

Materiel  Deficiency  Report 

Mission  Element  Need  Statement 

Military  Interdepartmental  Purchase  Request 

Memorandum  of  Agreement 

Manpower,  Personnel  and  Training 

Maintenance  Planning 

Management  System  Indicator 

Manpower  Requirements  and  Personnel 

Multiservice  Test  and  Evaluation 

Mobile  Training  Set 

NAVPRO 

NRTS 

Naval  Plan  Representative  Office 

Not  Repairable  This  Station 

B- 14 


NSARC 

NSN 

Navy  Systems  Acquisition  Review  Council 

National  Stock  Number 

OMB 

ORLA 

OSD 

OT&E 

Office  of  Management  and  Budget 

Optimum  Repair  Level  Analysis 

Office  of  the  Secretary  of  Defense 

Operational  Test  and  Evaluation 

PA 

P&A 

PAGEL 

PAR 

P&B 

PD 

P&R 

PCA 

PCI 

PCO 

PDM 

PDR 

PE 

PHT 

PI 

PM 

PMD 

PMEL 

PMP 

PMRT 

PMRTP 

P/N 

PO 

POM 

PPBS 

POT 

PR 

PRG 

PRR 

PT 

Program  Authorization 

Price  and  Availability 

Priced  AGE  List 

Program  Assessment  Review 

Planning  and  Budgeting 

Program  Director 

Planning  and  Resources 

Physical  Configuration  Audit 

Product  Configuration  Identification 

Principal  Contracting  Officer 

Program  Decision  Memorandum 

Preliminary  Design  Review 

Program  Element 

Packaging,  Handling  and  Transportation 

Program  Introduction 

Program  Manager 

Program  Management  Directive 

Precision  Measurement  Equipment  Lab 

Program  Management  Plan 

Program  Management  Responsibility  Transfer 
Program  Management  Responsibility  Transfer  Plan 
Part/Number 

Program  Office 

Program  Objective  Memorandum 

Planning,  Programming  and  Budgeting  System 
Preliminary  Qualification  Testing 

Purchase  Request 

Program  Review  Group 

Production  Readiness  Review 

Personnel  and  Training 

OA 

OAM 

QOT&E 

OT&E 

Quality  Assurance 

Quality  Assurance  Manager 

Qualification  Operational  Test  and  Evaluation 
Qualification  Test  and  Evaluation 

RDT&E 

RFP 

RFQ 

RIB 

R&M 

Research,  Development,  Test  and  Evaluation 
Request  for  Proposal 

Request  for  Quotation 

Recoverable  Item  Breakdown 

Reliability  and  Maintainability 
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SA 

SAC  SO 

SAF 

SAR 

SATAF 

SCT 

SD 

SDDM 

SDR 

SE 

SECDEF 

SERD 

SISMS 

SM 

SMR 

SOC 

SON 

SOW 

SPDR 

SPR 

SRP 

SRR 

SRU 

SS 

SSA 

SSAC 

SSEB 

SV 


Supplemental  Agreement 

Strategic  Air  Command  Support  Office 

Secretary  of  the  Air  Force 

Selected  Acquisition  Report 

Site  Activation  Task  Force 

System  Compatibility  Test 

Space  Division 

Secretary  of  Defense  Decision  Memoradum 
System  Design  Review 
Support  Equipment 
Secretary  of  Defense 

Support  Equipment  Requirements  Document 
Standard  Integrated  Support  Management  System 
System  Manager  "  A 

Source  Maintenance  Recoverability  Code 

Statement  of  Capability 

Statement  of  Need 

Statement  of  Work 

System  Program  Director's  Review 

Secretary  of  the  Air  Force  Program  Review 

Solicitation  Review  Panel 

System  Requirements  Review 

Shop  Replaceable  Unit 

Supply  Support 

Source  Selection  Authority 

Source  Selection  Advisory  Council 

Source  Selection  Evaluation  Board 

Survivability 


TACSO 

TCO 

TCTO 

TD 

T&E 

TEMP 

TH 

TO 

TPWG 

TWG 


Tactical  Air  Command  Support  Office 

Termination  Contracting  Office 

Time  Compliance  Technical  Order 

Technical  Data 

Test  and  Evaluation 

Test  and  Evaluation  Master  Plan 

Transportation  and  Handling 

Technical  Order 

Test  Planning  Working  Group 

Transfer  Working  Group 


UUT  Unit  Under  Test 


V&V 


Verification  and  Validation 


WBS 

WRSK 

WSEP 

WSMC 

WUC 


Work  Breakdown  Schedule 
War  Readiness  Spares  Kit 
Weapon  System  Evaluation  Program 
Western  Space  and  Missile  Center 
Work  Unit  Code 


B-16 


I 
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SUMMARY  OF  MAJOR  ACQUISITION  ACTIVITIES 
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APPENDIX  C.  SUMMARY  OF  MAJOR  ACQUISITION  ACTIVITIES 

This  appendix  contains  additional  detail  on  the  major  acqui¬ 
sition  activities  the  PM  must  manage.  There  are  three  parts  to 
this  appendix.  The  first  part  (pages  C-l  to  C-4 )  includes  a  set 
of  four  exhibits  showing  the  flow  of  activitiy  in  the  Concept 
Exploration,  Demonstration  and  Validation,  Full-Scale  Development 
and  Production  phases.  The  second  part  (pages  C-5  to  C-23  )  is  an 
annotated  listing  detailing  the  activities/events,  the  groups 
involved  and  descriptive  comments  regarding  the  activity  or 
event.  These  first  two  parts  are  taken  from  AFSCP-800-3 ,  A  Guide 
for  Program  Management,  currently  in  revision  and  no  longer  in 
print.  The  third  part  of  this  appendix  (pages'  C-24  to  C-30 )  is  a 
set  of  exhibits  illustrating  the  typical  organization  of  the 
various  directorates  in  a  typical  program  office.  These  have 
been  taken  from  the  Weapon  System  Acquisition  Guide. 
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AFSCP  800  3,  Air  Force  Systems  Command, 
FLOW  IN  PRODUCTION  PHASE 
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BLOCK  # 


ACTIVITY/EVENT 


PREPARED  BY 


Concept  Exploration  Phase 


1  •  Prepare  ROC  and  maintenance  concept 

2  •  Perform  mission  analysis 

3  •  AFSC  ROC  Review  and  Recommendation 

4  •  Air  Staff  Review,  ROC  Validation 

5  •  Prepare  Technology  Roadmap  (TR) 

•  Advanced  Development  Programs 

•  Issue  Program  Mgt.  Directive  ( PMD) 

•  Establish  program  priorities  and 
direction 

9  •  Consider  alternatives  and  proposal 

activities  -  includes  the  following: 


est.  program  office  (PO) 
initiate  operational  concept 

prepare  preliminary  design  (PD) 


evaluate  feasibility  and  risk 
assessment 

est.  production  feasibility 
assessment 


est.  logistics  support  estimate 
est.  intelligence  estimate 
est.  preliminary  test  estimates 
—  est.  cost/schedule  estimate 
perform  formal  tradeoff  studies 
perform  utility  cost/effective¬ 
ness  analysis,  LCC/DTC  strategy 


Oper.  Command s/MAJCOMs 
Oper.  Commands /MAJ COM s 

AFSC 

Air  Staff 

HQ  USAF 

HQ  USAF  receives  PMD  and 

AFSC  and  Oper.  Commands 

Oper.  Commands 
AFSC  and  contractors 


Surmarized  from  AFSCP-800-3 


* 


PREPARED  FOR  COMMENTS 


Submitted  to  AFSC 

initiates  CE  phase 

initiates  ROC;  used  during  review 
and  valid,  of  ROC 

HQ  USAF 

AFSC  submits  ROC  to  HQ  USAF  for  use 
to  validate  ROC  and  prepare  direc¬ 
tive  for  action 

HQ  USAF  and  OSD 

TR  info,  applied  to  ROC  before 

HQ  USAF  review?  info  affects  ROC 
input;  identifies  status  of  adv. 
develop,  programs  and  demo. 

implement  tasks  identified  in  TR 

translates  ROC  into  proposal  for 
new  program 

sends  it  to  HQ  AFSC 

HQ  AFSC  est.  program  priority  and 
issues  direction  to  AFSC  organiza¬ 
tions  via  AFSC  Form  56 

HQ  USAF 

info,  is  Used  to  prepare  program 
develop. /acquisition  plan,  advocacy 
documents  by  Air  Staff,  and  parts 
of  the  DCP 

AFSC 

-  should  be  done  early  in  CE 
phase?  info,  serves  as  basic 
input  for  PD 

-  compare  competing  alt.  system 
design;  info,  serves  as  input 
to  other  activ.  of  period 

-  provides  specific  D&T  activity 
for  demonstration  hardware 

in  Validation  Phase 

-  identifies  prod,  develop., 
tests  and  demo,  to  be  accom¬ 
plished  in  FSD  Phases 

See  AFSC P  800-3,  pp.  2-6  to  2-7 


Surmiarized  from  AFSCP-800-3 


BLOCK 

* 

ACTIVITY/EVENT 

PREPARED  BY 

PREPARED  FOR 

COMMENTS 

Concept  Exploration  Phase 

-  analyze  mgt.  and  procurement 
approaches 

consider  facility  support  esti- 
raate/site  survey 

10 

• 

Provide  guidance  and  support  to 
activities 

AFSC  monitors  activities 
of  AFSC  organizations 

11 

• 

USAF  advocacy  and  planning 

USAF  monitors  AFSC 
ef  forts 

12 

Informal  monitoring  by  OSD 

OSD  considers  proposed  system  in 
terms  of  DoD-wide  context 

13 

• 

Prepare  outline  for  draft  DCP 

OSD 

USAF 

if  outline  is  not  prepared  guidance 
for  draft  DCP  is  provided  to  AF? 

DCP  contains  decision-review  thres¬ 
holds  which  if  exceeded  cause 

SECDEF  program  review;  see  AFSCP 
800-3,  p.  2-8. 

14 

• 

Submit  planning  documents 

AFSC  Program  Mgr. 

HQ  USAF 

used  as  inputs  to  draft  DCP 

15 

• 

Arrange  Joint  Operational 
and  Technical  Review  (JOTR) 

HQ  AFSC 

HQ  USAF 

arranged  only  when  directed  by  the 
Commander,  AFSC,  see  AFSCP  800-3, 

p •  2  —8 . 

16 

• 

Prepare  draft  DCP 

HQ  USAF 

SAF 

17 

• 

Request  for  program  decision  draft 

DCP 

SAF 

OSD 

18 

• 

Update  PMD 

HQ  USAF 

HQ  USAF  publishes  directives  in  a 
revised  PMD  for  preparing  a  draft 
RFP  for  next  phase  (usu.  Valida¬ 
tion  ) 

19 

• 

Est.  program  priority  and  issue 
guidance  and  direction 

HQ  AFSC  receives  revised 

PMD  from  HQ  USAF 

See  AFSCP  800-3,  p.  2-8 

20 

• 

Est .  formal  PO 

See  AFSCP  800-3,  p.  2-8 

-  PM  should  select  and  have  his 

staff  by  product  division 
preparation  activities 

-  functional  baseline  (program  req'ts 

baseline)  is  est.  by  end  of  CE 
phase 

21 

• 

Consider  product  division  preparation 
activities 

See  AFSCP  800-3,  p.  2-9 

Summarized  from  AFSCPP-800-3 


4 


BLOCK  #  ACTIVITY/EVENT  PREPARED  BY  PREPARED  FOR  COMMENTS 


PMP  updated  and  submitted  as  master 
plan 

master  plan  implements  activities 
in  Block  9 

contents  of  PMP  such  as  SON,  RFP 
mgt.  plan,  source  selection,  adv . 
procurement,  product.,  test, 
logistics  support,  etc. 

PM  prepares  Systems  Command 
Information  Plan  (SCIP) 

prepare  D&F 


prepare  advanced  procurement  plan  .  PM 


review  of  RFP  by  AFSC  Proc.  Eval. 
Panel 

prepare  initial  TEMP 
-  production  planning 

O 

I 

-o 

22  •  Consider  HQ  AFSC  guidance  and  support 


Procur.  Eval.  Panel  (PEP)  may 
meet  at  HQ  AFSC  to  eval.  RFP’s 


23,24 

• 

Reviews  of  draft  DCP 

OSD  reviews  and 

23,24 

• 

Revision  of  DCP  to  reflect  OSD 
comments 

USAF 

23,24 

• 

Revised  DCP  becomes  "for  coordination" 
draft  used  as  basis  for  DSARC  action 

24  A 

• 

Initiate  informal  pre-DSARC  staff 
meeting 

24A 

• 

Consider  other  activities  preceding 
DSARC  ratgs 

-  DDR&E  ( T&E )  Test  and  Eval.  Report 

-  Chairman  of  CA1G  eval. 

24A 

• 

DSARC  review  meeting 

24  A 

• 

Prepare  stmt  of  objectives  and 
memorandum 

DSARC  Chairman 

See  AFSCP  800-3,  p.  2-9 


-  should  be  prepared  60  days 
before  RFP  release 

roust  be  prepared  prior  to 
releasing  RFP;  normally 
approved  by  Service  Secretary 
plan  is  revised  procurement 
approach  of  that  est.  in 
Block  9 

done  only  when  AFSC  Form  56 
transmitting  the  PMD  so 
specifies 

should  be  done  in  CE  Phase 

-  should  emphasize  resolution  of 
production  risks  identified 

in  Block  9 

support  provided  in  preparing 
SOW  and  RFP 


comments  on  DCP 
OSD 


not  required,  but  should  precede 
DSARC  review  by  60  days 


SEC DEF  provided  within  15  days  after 

DSARC  review 


BLOCK  # 


ACTIVITY/EVENT 


PREPARED  BY 


Concept  Exploration  Phase 


24B  •  SECDEF  review  of  DCP 

24C  •  SECDEF  aproval  of  DCP 


Demonstration  &  Validation  Phase 


25  •  Approved  DCP  transmitted  by  SAF 


26 


Issue  priorities,  directives,  hq  USAF 

Budget  Auth . /Procure .  Auth.  (BA/PA) 


O  26A  •  Develop  initial  drafts  of  prog.  pm  &  PO 

qq  supplements  to  PMD 

27  •  Issue  AFSC  priorities,  guidance  and  HQ  AFSC 

direction 

28  •  Prepare/issue  PMP  documents  which 

include  the  following: 

-  revise  and  supple,  req'ts  bsline 
doc.  prepared  in  Block  22 

28A  •  Prepare/issue  PMP  PM 


•  Initiate  validation  program: 

-  est.  realistic  perform,  specif, 
(alloc,  bsline)  which  meet  the 
O&S  req'ts. 

est.  schedule  and  cost  estim  for 
FSD  phase 

-  est.  planning  schedules  and  cost 
estim  for  product. 

-  achieve  RFP  release 
develop  procurement  approach 


0 


Summarized  from  AFSCP-800-3 


PREPARED  FOR 


COMMENTS 


constitutes  program  initiation 
decision  approved  DCP  signed  by 
SECDEF  published  within  30  work¬ 
days  after  SECDEF  decision  is 
made;  begin  Demo.  &  Val .  Phase 
forwarded  to  HQ  USAF  with  funding 
req'ts  consistent  with  FYDP 


AFSC  BA  issued  for  ea .  FY  incre¬ 
ment  (or  entire  FY  program  see  - 
AFSCP  800-3,  p.  3-1. 


PO 


See  AFSCP  800-3,  p.  3-5 


PM  is  responsible  for  overall 
preparation  and  issuance  of  PMP 
with  coordination  from  AFLC,  ATC, 
AFOTEC  and  the  operating  command; 
principal  program  ragt.  baseline 
document;  should  be  kept  up- 
to-date;  tailored  to  needs  of 
the  program 


contractors 


develop  this  early  in  acquisition 

process  consider  contrary  types  - 


BLOCK  ♦ 


activity/event 


PREPARED  BY 


Demonstration  &  Validation  Phase 


•  Consider  Validation  by  competitive 
contactors  DoD-financed;  under  PM*s 
guidance 

•  Consider  Validation  by  sole-source 
contractors 


29  •  Consider  evaluation  of  proposals  bv 

SSAC  1 

29  •  Initiate  source  selection  and  contract 

A/B/C  award 


30 


O 

I 

V£> 


30 

30 


•  Implement  validation  program 

•  Transition  from  functional  to  allocated 
bsline 

•  Initiate  DT&E  and  prepare  T&E  Master 
Plan  AFSC  test  center  and  PM 


•  Develop  procurement  plan  -  consider 
the  following: 

incentive  provisions 

-  procureraent/production  leadtimes 
valid*  of  cost  and  price  estimates 

-  logistics  req'ts 

-  equip,  and  spares  req*  ts 
facility  req'ts 

-  pers .  and  training  req'ts 

•  Est.  allocated  bsline  (design  req'ts 
bsline) 


•  Produce  system  definition 


•  Begin  contractor  FSD  PMP  specified  by  PO 

•  Consider  design  standardization 


Suntnarized  from  AFSCP-800-3 


4 


PREPARED  FOR 


COMMENTS 


4 


Valid.  Phase  may  be  on  sole— source 
basis  if  competition  is  not 
feasible-decision  is  made  by  SAF 
based  on  PM  recommendation 


procedures  described  in  AFM  70-6 
and  AFR  70-15  may  be  used  for 
any  Valid.  Phase  approach  in 
Air  Force 


See  AESCP  800-3,  p.  3-7 


DT&E  of  subsystems  begins  in 
valid.  Phase-feasibility  testing 
should-  beg  in  earlier;  check  to 
insure  that  design  is  functional 


should  satisfy  objectives  of  func¬ 
tional  bsline  (program  req'ts 
bsline)  see  AFSCP  800-3,  p.  3-7 

Valid.  Phase  should  produce  a  more 
detailed  system  definition  as  func¬ 
tional  bsline  grows  into  allocated 
bsline 


See  AFSCP  800-3,  p.  3-8 


01- 


Summarized  from  AFSCP-800-3 


BLOCK  #  ACTIVITY/EVENT  PREPARED  BY  PREPARED  FOR 

Demonstration  &  Validation  Phase 


COMMENTS 


32  •  Initiate  prototype  demonstration  operating  command  participates 

33  •  Initiate  hardware  proofing  decision  made  by  PM 

34  «  Perform  tradeoff  studies 


35  •  Perform  risk  assessment  (should  be 

done  continually) 


O  36  •  Evaluation  validation  effort  which 

includes  operating  commands  partici¬ 
pate  the  following; 

design  to  cost  goals  (when 
required  by  PMD )  must  be 
produced  during  this  phase 
and  expressed  as  cost  ceilings 
identify  performance  and  schedule 
trade-offs 

-  hardware  evaluation 
cost  estimating 
production  risk  evaluation 
ATC  participation 

evaluation  for  FSD  which  includes 
a  PRR 


•  Initiate  document  preparation  which 
includes  x 

-  update  DCP,  PMP,  Adv.  Procure. 
Plan 

-  contracts  for  FSD  and  production 
phases  should  generally  be 
separate 


degree  of  prototyping  and  number 
of  system  segments,  subsystems  to 
be  prototyped  should  be  defined 
in  Valid.  Phase  adv.  procure, 
plan.  See  AFSCP  800-3,  p.3-9 


studies  ensure  that  configuration 
being  defined  for  FSD  is  balanced 
among  total  cost,  schedule  and 
operational  effectiveness;  changes 
to  any  of  above  require  DCP/DSARC 
procedure;  see  AFSCP  800-3,  p.  3-9 

should  identify  and  order  elements 
of  risk  which  constitute  the  most 
important  uncertainties  in  FSD 
phase 


if  required,  should  be  planned 
for  in  Block  3 
results  of  this  review  are 
reflected  in  RFP  for  FSD  and 
in  approval  documentation 
for  Ratification  Decision 


r 


* 


IT-: 


BLOCK  #  ACTIVITY/EVENT 

Demonstration  &  Validation  Phase 

38  •  Submit  updated  DCP 

39  •  Request  for  Ratification  Decision 

40  •  Update  DCP 

41  •  DSARC  II 

42  •  SECDEF  approval  of  DCP 


•  Identify  required  changes  to  funding, 
schedules,  and  technical  planning  that 
will  effect  Production  Phase 


PREPARED  BY 


HQ  USAF 


SAF  reviews  program 


PREPARED 

SAF 

DSARC 


FOR 


Summarized  from  AFSCP-800-3 
COMMENTS 


4 


* 

PM  cannot  influence  time  it  takes  b 
get  Ratification  Decision  DCP 
approved?  DCP  must  be  kept  current 


DCP  is  updated  to  satisfy  req'ts 
of  FSD 

DSARC  reviews  updated  DCP  and 
other  prog.  doc. 

approval  of  DCP  constitutes  the 
Ratification  decision;  decision 
depends  on  confirmation  of  tech- 
technical,  financial  and  schedule 
constraints  because  of  the  Validation 
Validation  Phase,  the  Air  Force 
will  make  one  of  the  following 
recommendations  to: 

(1)  contract  for  full-scale 
development . 

(2)  Continue  further  Validation 
Phase  effort. 

(3)  Defer  or  abandon  the  develop¬ 
ment  effort. 

(4)  Undertake  further  exploratory 
or  advanced  development  of  key 
components  and/or  system 
studies . 

Approval  into  FSD  is  based  on: 

(1)  System  tradeoffs  have  produced 
a  balanced  and  realistic  set  6 
performance  parameters. 

(2)  Risk  areas  have  been  identi¬ 
fied  and  reduced  to  acceptable 
levels. 

(3)  COst/schedule  estimates  for 
full-scale  development  are 
acceptable. 

(4)  Contractual  aspects  are  sound 
(terms  and  conditions  are 
appropriate  to  risk  and 
funding  related  to 
milestones) . 
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BK  # 

ACTIVITY/EVENT 

PREPARED  BY 

Full  Scale  Development  Phase 

43 

• 

Transmit  approved  DCP 

SAF  forwards  DCP  to 
HQ  USAF 

44 

• 

Issue  HQ  USAF  directives,  BA/PA 

HQ  USAF  forwards  DCP 
to  AFSC 

45 

• 

Issue  AFSC  guidance  and  direction 

46 

• 

Award  contract  according  to  approved 
DCP 

46 

A/B/C 

• 

Initiate  Source  Selection  Review 

47 

• 

Revise  management  baseline 

o 

48  •  Initiate  design  activity  contractor 


49  •  Conduct  Verification  Reviews 


50 


•  Develop/demonstrate  production 
techniques 

•  Update  documentation  for  development 


Summarized  from  AFSCP-800-3 


PREPARED  FOR 


COMMENTS 


HQ  USAF  releases  funds  for 
Development  Plan  through  BA/PA 
actions 


See  AFSCP  800-3,  p.  4-1 


source  selection  documents  are 
submitted  to  the  designated  level 
for  review  and  decision  making 

changes  in  mgt.  plans  and  documen¬ 
tation  are  completed  to  make 
program  plans  compatible  with 
approved  FSD  phase  and  FSD  DCP 

this  represents  the  iterative 
design  effort  accomplished  by 
the  contractor  during  FSD?  major 
effort  of  preliminary  detail 
design  leading  to  developing  an 
acceptable  design  approach  should 
begin  at  this  point?  reduce  pro¬ 
duction  risks  and  prepare  for 
Final  PRR 

PDR  for  ea.  Cl  is  conducted  before 
beginning  detailed  design  process; 
AF  position  on  design  is  est.  as 
result  of  Verification  Reviews?  PM 
can  request  the  use  of  Independent 
Technical  Audit /Assessment  per¬ 
formed  at  his  discretion 

consider  using  the  "Fly-Before  Buy" 
approach 

the  following  doc.  require  updating 
during  the  early  stages  of  FSD: 

a.  Logistic  Support  Plan 

b.  Training  Plan 

c.  Advanced  Procurement  Plan 

d.  Production  Plan 

e.  Subcontract  Mgt.  Plan 


Surmiarized  from  AFSCP-800-3 


BLOCK  # 


ACTIVITY/EVENT 


PREPARED  BY 


PREPARED  FOR 


COMMENTS 


51  •  Continue  DT&E  which  includes: 

contractor  DT&E  contractor 


-  AF  DT&E 


AF 


f.  PMRT  Agreement 

g.  Turnover  Agreement 

h.  to  Mgt.  Plan 


during  FSD  contractor  should 
expand/ revise  TEM  test 
normally  consists  of  testing 
Cl  *  8  and  usu •  completed  before 
PCA;  plan  and  procedures 
should  follow  QA  and  system 
specif,  req'ts 
during  FSD  AF  should 
expand/revise  TEMP;  proce¬ 
dures  should  implement  QA 
and  system  specific  req'ts? 
also  emphasize  integrated 
eval.  of  system  segments 
required  for  mission 


O  -  Est.  PMRT  date 

I 

H* 

U> 


implementing  command  PM  forwarded  to 
and  supporting  command  SM  HQ  USAF 


-  Logistics  Support  Plan 
Advanced  Procurement  Plan 
Production  Plan 

-  Training  Plan 

52  •  Begin  FCA 


53  •  Begin  Verification  Reviews 


CDR  should  occur  before  start  of 
DT&E/IOT&E 

PRR  should  be  coonduct  at  this  performed  by  product, 

point  in  the  Development  Phase  element  in  PO 

other  design  and  program  reviews 
should  be  held  at  significant 
milestones  to  confirm  accom¬ 
plish.  and  eval.  tech,  progress 
before  production  phase 


during  FSD  the  respective  PM  & 
SM  jointly  est.  PMRT  date 
( PMRTD) ?  formal  PMRTD  is 
forwarded  to  HQ  USAF  for 
inclusion  in  the  production 
PMD;  PMRT  will  occur  at  the 
earliest  possible  date  in 
the  production  phase: 


FCA  begins  when  DT&E  completed? 

FCA  is  complete  when  Cl  is 
qualified 

qualif.  of  USAF  equip  (GFAE,  sub¬ 
systems)  may  be  given  at  this  time 
or  withheld  until  after  the  verifi¬ 
cation  reviews  and  tests  of 
Product.  Phase 


n-; 


BLOCK  # 


ACTIVITY/EVENT 


PREPARED  BY 


54  •  Evaluate  development  testing 

55  •  Monitor  program 

56  •  Review/redirect  program  (DCP 

thresholds) 

57  •  Update  program  documentation 


58/59  • 

60  • 


Review/subrait  updated  DCP 
Request  production  decision 


HQ  AFSC 


HQ  USAF 


Summarized  from  AFSCP-800-3 


PREPARED  FOR 


COMMENTS 


this  activity  shows  evaluation  of 
testing  completed  in  FSD  phase? 
make  any  necessary  engineering 
changes 

HQ  AFSC  monitors  the  program  durinq 
FSD 

technical  direction  from  higher 
levels  is  not  expected  during  FSD 
unless  thresholds  are  broken  or 
threatened 

product  identification  baseline 
should  be  as  complete  as  possible 
for  the  production  contract  RFP, 
even  though  updating  will  continue 
until  the  PCA;  the  majority  of 
DTfirE,  including  some  IOT&E,  if 
schedule  permits,  should  be  com¬ 
pleted  before  the  product  baseline 
is  established  to  avoid  the  cost  of 
processing  formal  engineering 
changes  required  from  DT;  product 
bsline  is  placed  at  end  of  qualifi¬ 
cation  testing;  before  award  of 
production  contract  a  plan  for 
turnover  to  oper.  command  and  PMRT 
identifi.  doc.  to  AFLC  should  be 
formalized- should  be  done  early 
enough  to  provide  for  programming 
and  funding  req'ts;  prepare  inde¬ 
pendent  cost  estimates. 


SAF 


Approval  to  proceed  into  the 
production  phase  will  be  based 
upon  assurance  that* 

a.  risks  (including  threat)  have 
been  resolve^ 

b.  costs,  schedule  and  perfor¬ 
mance  estiraates/req • ts  for 
production  phase  are  credible 

c.  certification  by  credible  DoD 
component  that: 

1)  practical  engineering 
design  has  been  completed 

2)  tradeoffs  have  optimized 
product,  mtn.  and  oper. 
cos  ts 


si-; 


o 


BLOCK  #  ACTIVITY/ EVENT 


61  •  Update  DCP 

62  •  DSARC  III  decision 

63  •  SECDEF  approval  of  DCP 


SuiTinarized  from  AFS^P-800-3 


PREPARED  BY 


PREPARED  FOR 


COMMENTS 


SECDEF  decision  after  consultation  with 
DSARC 


3)  contractual  aspects  are 
sound 

DCP  is  updated  in  preparation  for 
the  production  phase 

program  doc.  should  be  checked 
against  DSARC  checklist  (see  APSCP 
800-3,  Attachment  1)  to  determine 
if  all  aspects  of  FSD  phase  have 
been  completed. 

SECDEF  approval  of  DCP  initiates 
Production  Phase 


•  There  are  two  distinct  periods  of  pro¬ 
duction: 

1. )  first  segment  immediately  follows 

FSD,  where: 

-  hardware  manufact.  is  at  peak  rate 

-  hardware  changes  from  FSD  and  early 
prod  testing  are  implemented 

2. )  second  segment  implements  follow-on 

production  after  peak  rate  is  achieved. 


Production  and  Deployment  Phase 


64  •  Transmit  SAF-approved  DCP 


65  •  Issue  USAF  priorities,  directives  and  HQ  USAF 

BA /PA 


-  obtain  program  element  numbers  HQ  USAF 

for  OSD  production  fund  scheduling 

66  •  Issue  AFSC  priorities,  guidance  and  HQ  USAF 

direction 

67  •  Imitate  Source-Selection  review 
67A/B 


67C  •  Consider  contract  award 


68  •  Consider  production  mgt  and  QA  mgt 


Transmitted  to  HQ 
USAF  USAF 

HQ  USAF  deter ./reaffirms  program 
priories,  issues  direction  and 
releases  program  funds  for  Pro¬ 
duction  Phase  through  BA/PA 
actions 

OSD  -  done  after  program  decision 


source-selection  doc.  or  sole- 
source  recommendation  preceeded 
by  contract  negotiations,  are 
submitted  for  review  and  decision  * 

doc.  prepared  during  FSD  for 
Production  Phase  are  revised  to 
reflect  DCP  guidance 


91 


Suimiarized  from  AFSCP-800-3 


O 


BLOCK  # 


ACTIVITY/EVENT 


PREPARED  BY  PREPARED  FOR 


COMMENTS 


68A 


Consider  follow-on  production 


69  •  Initiate  verification  reviews 


70 


•  Continue  Physial  Configuration 
Audit  ( PCA ) 


71  •  Continue  DT&E/lOT&E 

72  •  Initiate  FOT&E 


•  Consider  reviews  and  redirection 
due  to  changes  in  cost,  perfor¬ 
mance,  schedule 

•  -  Begin  formal  turnover  of  facilities 


•  Continue  OT&E  for  deficiencies  and 
modifications 


•  ,  Begin  Produce  Management  Respon¬ 
sibility  Transfer  ( PMRT) : 


this  initiates  second  segment  of 
production,  review  previous  pro¬ 
duction  decisions 

reviews  should  continue  on  a 
periodic  basis  to  assure  that 
design  continues  to  be  feasible 
and  sound;  CDR  is  completed  be¬ 
fore  DCA 

PCA  is  a  formal  audit  that  com¬ 
pares  Part  II  Detail  (Produce 
Fabrication)  Specif,  and  accom— 
paning  drawings  against  product, 
hardware;  also  deter,  that  accep¬ 
tance  test  req'ts  are  adequate; 
product  of  PCA  is  PM  acceptance 
of  above  specific;  PCA  is  pre¬ 
requisite  to  configuration  item 
acceptance;  successful  completion 
of  PCA  est.  approved  product  con¬ 
figuration  bsline  for  Cl  and 
begins  formal  engineering  change 
control  for  class  I  design 
changes 

continue  testing  from  FSD  phase 

should  be  conducted  on  early 
product,  mode  or  after  the  com¬ 
mand  has  accepted  the  first 
operating  unit  and  its  updated 
changes 

technical  direction  from  higher 
levels  is  not  expected  during 
production 


AFTEC  or  Oper .  Commands  evaluates  OT&E 
results;  PM  should  have  authority  within  the 
program  to  decide  on  updates;  if  modifica- 
tions  are  required  HQ  USAF  should  designate 
implementing  command 


turnover  occurs  when  oper.  com¬ 
mand  accepts  responsibility  for 
operation  and  ratn  of  new  system; 
details  of  turnover  are  set  forth 
in  Formal  Turnover  Agreement 

complete  logistical  support  is 
required  to  test  the  system  in  an 
operational  environ. 


termination  of  AFSC  PM  responsibility;  AFLC 
assumes  logistics  and  mgt.  responsibility 


PMRT  planning  begins  early  in 
FSD  see  AFSCP  800-3  pp.  5-6  to 


BLOCK  * 


ACTIVITY/ EVENT 


PREPARED  BY 


-  PMRT  should  be  scheduled  and 
reported  as  a  major  milestone 

-  PMRT  date  should  be  negotiated  by 
the  PM  w/AFLC/ALC  SM 

-  PMRT  date  should  be  early  in  the 
product,  phase  and  after  comple¬ 
tion  of  DT  when  design  has  stabil¬ 
ized 

-  PO  should  be  dissolved  after  PMRT 
and  replaced  w/Project  Office 

•  Begin  deployment  planning 


Program  Control  Considerations 


•  Consider  various  types  of  schedules! 

“  process  charts 

-  Gantt  charts 

-  milestone  charts 

-  line  of  balance  (LOB)  technique 

-  networking 

•  Consider  planning  which  should  be  done: 

-  when  interdependent  activities 
are  involved 

-  to  ensure  quality  decisions 

•  Est .  relationship  w/external  activi¬ 
ties  such  as  OCAS,  AFPROs,  etc. 


•  Determine  program  direction  HQ  USAF 

•  Determine  organization  of  program 
control  element  in  the  program 
office 

•  Review  of  AFSC  acquisition  program 
at  various  milestones  or  when 
warranted  which  includes: 

-  Commanders  Daily  Staff  Mtg 

-  Mission  Mgt.  Review 


Executive  Mgt.  Review 
Staff  Presentations 


Summarized  from  AFSCP-800-3 


PREPARED  FOR 


Command  reviews 
program  status 


COMMENTS 


5-7 


consider  support  required  for 
newly  fielded  system  (i.e.,  mtn, 
training);  est.  feedback  pro¬ 
cedures  for  field  generated  data 
needs  of  PO  deter.,  type  of  sched- 
dule  used 


see  AFSCP  800-3  pp.  6-9  tp  6-10 


rhould  be  done  early  in  program's 
life  and  confirmed  in  writing  by 
Memorandum  of  understanding 


see  pp  6-1  to  6-2 


see  Fig.  6-3  (from  AFSCP  800-3) 
for  more  detail 

-  performed  daily 

-  monthly  status  report  of  Develop. 
Planning,  etc.  ea.  reviewed 
quarterly 

-  monthly  status  report 

-  done  as  required 
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BLOCK  # 


activity/event 


PREPARED  BY 


O 


Program  Assessment  Review  (PAR) 

SAF  Program  Review  (SPR) 

Command  Assessment  Review  (CAR) 

Field  Assessment  Review  (FAR) 
Mgt .  Assessment  Review  (MAR) 


•  Consider  roles  of  -  SYSTO 


-  DCASO/AFPRO 

-  PEM 


Engineering  Management  Considerations 


•  Consider  engineering  mgt  should  be 
used  in  all  acquisition  phases 

•  System  engineering  mgt  should  be  used 
in  all  acquisition  phases 

•  Consider  Technical  Reviews 

•  Consider  system  Req'ts  Review  ( SRR) 

•  Consider  System  Design  Review  (SDR) 


•  Consider  Preliminary  Design  Review 
( PDR) 


•  Consider  Critical  Design  Review  (CDR) 


•  Consider  Functional  Configuration 
Audit  ( FCA) 


Summarized  from  AFSCP-800-3 


PREPARED  FOR 


COMMENTS 


-  quarterly  status  of  ea .  major 
system  program 

-  updated  monthly  between  quarters 

-  each  program  reivewed  quarter¬ 
ly?  CARs  are  held  weekly 

-  held  semi-annually 

-  review  by  product  division 
Commander  of  program  involving 
more  than  $2M  to  complete?  not 
review  by  AFSC  unless  nominated 
for  CAR 

SYSTO  is  POC  at  HQ  AFSC 

DCASO/AFPRO  administers  contracts 

PEM  is  POC  w/HQ  USAF 


process  for  managing  the  engineer¬ 
ing  effort  should  be  uniform 

see  pp.  8-3  to  8-4  for  details 


number  and  type  of  reviews  depend 
on  complexity  of  the  aquisition 

see  AFSCP  800-3  p.  8-5 

should  be  conducted  as  final 
review  before  submittal  of  valid 
phase  products  or  as  intial  FSD 
review  for  systems  not  requiring 
formal  valid  phase;  if  SDR  is 
satisfactory  functional  beline 
should  be  updated  and  contract- 
tor  enters  into  prelim*  design; 
see  AFSCP  800-3  p.  8-5 

should  be  conducted  for  ea.  Cl  in 
system;  successful  PDR  is  re¬ 
quired  for  ea.  Cl  before  pro¬ 
ceeding  into  detail  design,  see 
AFSCP  800-3  p.  8-5 

review  should  be  conducted  before  $ 

design  is  committed  to  produc¬ 
tion;  see  AFSCP  800-3  p.  8-5 


Vi 


see  AFSCP  800-3  p.  8-5 


BLOCK  # 


ACTIVITY/EVENT 


•  Consider  Physical  Conf iguraiton  Audit 
(  PCA) 

•  Consider  Formal  Qualification  Review 
(FQR) 


•  Consider  Technical  Interchange/ 
Direction  Meetings  (Tl/D) 

•  Consider  Independent  Reviews 

•  Consider  decision  risk  analysis 

•  Initiate  Technical  Performance  Measure- 
ment  (TPM) 


n 

» 

H* 

VO 


•  Consider  reliability  during  acquisi¬ 
tion  which  includes  the  following: 

-  perform  ROC  analysis 

-  update  and  refine  system  anlaysis 

numerical  reliability  req'ts 

-  reqt's  allocation 

part  selection  and  standardization 

-  design  analysis  and  prediction 

-  design  reviews 

-  demonstration  (qualif.)  testing 

quality  assurance 

R&M  production  (verification) 

testing 

R&M  update  changes 

•  Consider  maintainability  program 


•  Consider  procedures  in  mtn.  analysis: 

level  of  repair  analysis  (LORA) 
maintainability  apportionment 
“  failure  mode  analysis 

•  Develop  detailed  maintenance  plan 


Sunnarized  from  AFSCP-800-3  * 

PREPARED  BY  PREPARED  FOR  COMMENTS 

see  AFSCP  800-3  p.  8-6  ’ 

at  the  FQR,  the  Cl  is  certified 
for  entry  into  the  gov't  inven¬ 
tory;  see  AFSCP  800-3  p.  8-6 

at  direction  of  PM 


see  AFSCP  800-3  p.  8-7 

initiated  during  Valid  Phase 
after  design-to  req'ts  of  the 
product  elements  are  defined  and 
continues  throughout  FSD  into 
Production  and  Deployment  Phases 


-  done  during  Conceptual  Phase 

-  done  during  Valid  Phase  to 
Conceptual  Phase  Info 

done  during  FSD  when  reliability 
activity  is  at  its  highest  level; 
see  AFSCP  800-3  pp.  8-9  to  8-10 


done  during  Production  Phase; 
see  AFSCP  800-3  p.  8-10 


should  be  coordinated  with  AFLC 
throughout  the  development  pro¬ 
gram;  mtn.  concept  and  design 
should  stabilize  by  FSD;  a  main¬ 
tainability  demonptration  is 
necessary  at  the  end  of  FSD 


this  will  be  incorporated  in 
the  ILS  plan  developed  by  AFLC 


oz-; 


BLOCK  * 


ACTIVITY/EVENT 


PREPARED  BY 


Configuration  Management  Considerations 


•  Consider  configuration  mgt. 

•  i  »  * 

•  Consider  Baseline  Mgt.  Three  base¬ 
lines  are  normally  est.  These  ares 


a.  Functional  Baseline 

b.  Allocated  Baseline 

c.  Product  Baseline 


O 


©  Consider  Class  I  vs.  Class  II  changes 


Test  and  Evaluation  Considerations 

•  Consider  role  of  PO  in  T&E 

•  Consider  Critical  Questions  and  Areas 
of  Risk  in  testing 

•  Update  test  plans 


Other  Program  Management  Considerations 


•  Consider  PRR*s 


Summarized  from  AFSCP-800-3 


PREPARED  FOR 


COMMENTS 


applied  during  life  cycle  of 
systems  and  all  Cl 

a  fundamental  concept  of  engin¬ 
eering  mgt  is  the  use  of  a  series 
of  technical  baselines  to  ensure 
an  orderly  transition  from  one 
major  decision  point  to  the  next 
throughout  the  Acquisition  Life 

a.  normally  est.  at  end  of  Con¬ 
ceptual  Phase  and  is  Concur¬ 
rent  w/ initiation  of  Valid. 
Phase 

b.  normally  est.  at  end  of  Valid. 
Phase 

c.  normally  est.  before  or  at  PCA 
during  the  latter  part  of  FSD; 
see  also  AFSCP  800-3  p.  9-2 
and  Fig  9-1 

Class  I  changes  effect  contract¬ 
ually  specified  items  and  must  be 
approved  by  the  CCB  chairman; 

Class  II  engineering  changes  must 
be  reviewed  by  plant  representa¬ 
tive  for  concurrences  in  classi¬ 
fication 


see  Fig.  10-2  from  AFSCP  800-3 

consider  these  areas  as  discussed 
in  the  PMO* 

when  TEMP  is  updated  due  to  design 
changes,  detailed  test  plans 
must  also  be  updated 

•e  >*  t  -**i  •  5 


should  be  accomplished  and  docu¬ 
mented  before  production  decision; 
preparation  for  PRR  should  be 
initiated  at  inception  of  design; 
long  leadtime  items  may  require 
incremental  PRR’s 


t 


4 


Sunroarized  from  AFSCP-800-3 


4 


BLOCK  # 


ACTIVITY/EVENT 


PREPARED  BY  PREPARED  FOR 


COMMENTS 


•  Consider  production  feasibility 
assessment 


•  Consider  ILS  planning  which  includes: 

-  develop  ILS  plan  PO,  ATC ,  AFLC,  Oper .  Comm. 


conduct  LSA  (mil.  std.  1388) 


•  Consider  training  and  training  equip¬ 
ment  planning 

•  Consider  interface  ragt.  during  all 
acquisition  phases 

•  Consider  data  during  acquisition  cycle 


•  Consider  possibility  of  Security 
q  Assistance  Programs  (SAP) 

I 

e  Consider  Program  office  organ. 

listed  below  is  various  info  relating 
to  a  PO 

est.  program  Cadre 


relationship  between  organ,  ele¬ 
ments  for  program  mgt  and  DSARC 

est.  formal  program  office 


consider  various  types  of  PO 
organization 


consider  responsibilities  of: 
Program  Control 
Configuration  Mgt 
Procurement  Function 
Production  Mgt 


should  be  accomplished  in  the 
Conceptual  Phase  of  develop, 
cycle  before  program  decision 


milestones  are  est.  to  coincide 
with  other  important  system 
milestones 

may  be  required  as  part  of  the 
RFP 

should  be  considered  during  all 
acquisition  phases 

see  AFSCP  800-3  pp.  15-1  to  15-3 


CDRL  is  introduced  (see  AFSCP 
800-3  p.  16-1) 

see  AFSCP  800-3  chap.  19 


see  AFSCP  800-3  chap.  20 


done  early  in  Conceptual  Phase 
phase  in  appropriation  AFSC 
AFSC  Product  Division  accord¬ 
ing  to  direction  rec'd  from 
HQ  AFSC  and  HQ  USAF?  cadre 
prepares  DCP,  PMP  etc.;  see  Fig. 
20-2 

see  Fig.  20-1  from  AFSCP  800-3 


achieved  after  Conceptual  Phase 
when  a  new  or  revised  PMD  direc¬ 
tion  entry  into  Valid.  Phase 
(or  later  phase)  is  issued 

see  Figs.  20-3  to  20-7  (from 
AFSCP  800-3);  PO  is  organized  as 
a  mission  element  in  one  of  the 
AFSC  product  Divisions 

see  AFSCP  800-3  pp.  20-11  to 
20-14 


zz- 
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BLOCK  #  ACTIVITY/EVENT  PREPARED  BY 


Engineering  Function 

T&E 

ILS 

Mgt  Support 

MAJCOM  Liason  Offices 

•  Synchronic©  manpower  req'ts 


•  Consider  use  of  METs 


•  Phaseout  of  PO  which  includes  5 

transfer  of  rag t .  responsib.  from 
AFSC  to  AFLC 

-  providing  for  operational  life 
engineering  support  req'ts 


•  Consider  deployment  aspects 


•  Incorporate  deployment  oriented  tasks 
and  schedules  into  PMP 


•  Consider  PMRT  date 


Surtttnarized  from  AFSCP-800-3 


PREPARED  FOR  COMMENTS 


manpower  req'ts  should  be  syn¬ 
chronized  with  program  milestone 
objectives;  PMP  should  contain 
organiz.  and  manpower  data  for 
both  acquisition  and  operation¬ 
al  phases ,  This  info,  is  impor¬ 
tant  in  Valid.  Phase  to  secure 
additional  manpower  authorizations 

aid  Project  Director  in  estimat¬ 
ing  the  manpower  needs  of  his 
program  during  acquisition 


AFSC  is  responsible  for  all  system 
engineering  during  conceptual. 

Valid,  and  Product  Phases?  AFLC 
is  responsible  for  operational 
engineering  of  the  system  during 
deployment  phase?  AFLC  will 
assume  operational  engineering 
responsibility  for  those  items 
officially  accepted  by  the  oper. 
command  and  have  become  part  of 
operational  inventory;  AFSC 
retains  overall  responsbility 
for  system  engineering  decisions 

address  deployment  aspects  to  the 
maximum  extent  practicable  during 
the  Conceptual  and  Valid.  Phases? 
planning  should  be  est.  by  PM 
with  representatives  of  partici¬ 
pating  organiz.  sarly  in  FSD 

deployment  oriented  tasks  and 
schedules  should  be  est.  as 
program  rogt.  milestones 

for  those  programs  directed  by 
PMD,  date  for  PMRT  is  determined  • 

by  AFSC  and  AFLC  during  FSD  phase 
and  forwarded  to  HQ  USAF  for  in¬ 
clusion  in  production  PMD-once 
est.  PMRT  data  cannot  be  changed 


* 


zz- 


activity/event 


PREPARED  BY 


BLOCK  # 


O 


Consider  turnover  problems 

Attachment  4  of  AFSCP  provides  info, 
on  "How  to  Prepare  a  PMP" 


Suranarized  from  AFSCP-800-3 


PREPARED  FOR 


COMMENTS 


except  by  HQ  USAF  based  on  a 
joint  recommendation  for  the 
change  by  Commanders,  AFSC  and 
AFLC;  for  those  programs  not 
directed  by  PMD,  PMRT  is  est.  by 
a  joint  command  document  -  est. 
date  may  not  be  changed  except 
by  agreement  between  Conmanders, 
AFSC  and  AFLC.  As  PMRT  date 
approaches  a  jointly  coordinated 
agreement  is  prepared  listing 
residual  tasks  to  be  performed 
by  AFSC  and  schedule  for  their 
completion  -  this  schedule  is 
forwarded  to  HQ  USAF  30  days 
before  est.  PMRT  date  for  review, 
approval,  and  issuance  by  PMO 

see  AFSCP  800-3  pp.  21-7  tp  21-8 


Source : 
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Weapon  System  Acquisition  Guide,  (Air  Command  and  Staff  College,  May  1981) 


Exhibit  C-5.  TYPICAL  ENGINEERING  DIRECTORATE  ORGANIZATION 
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CONFIGURATION  MANAGEMENT 
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Source:  Weapon  System  Acquisition  Guide,  (Air  Command  and  Staff  College,  May  1981) 

Exhibit  C-6 .  TYPICAL  CONFIGURATION  MANAGEMENT  DIRECTORATE 
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Source:  Weapon  System  Acquisition  Guide,  (Air  Command  and  Staff  College,  May  1981) 

Exhibit  C-7 .  ORGANIZATION  OF  A  TYPICAL  PROGRAM  CONTROL/BUSINESS 

MANAGEMENT  DIRECTORATE 
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DIVISION  CHIEF 


CONTRACTING 

OFFICER 

(FMS) 

CONTRACTING 

OFFICER 

(AIRFRAME) 

CONTRACTING 

OFFICER 

(CHANGES) 
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OFFICER 

(GFE) 
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Source:  Weapon  System  Acquisition  Guide,  (Air  Command  and  Staff  College,  May  1981) 


Exhibit  C-8 


TYPICAL  PROGRAM  OFFICE  CONTRACTING  DIVISION 


83- 


Source:  Weapon  System  Acquisition  Guide,  (Air  Command  and  Staff  College,  May  1981) 


Exhibit  C-9. 


TYPICAL  PROGRAM  OFFICE  MANUFACTURING  DIVISION 

NOTE:  MATRIXED  FROM  A  REMOTE  MANUFACTURING  DIRECTORATE 
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to 

DIRECTORATE 

FOR 

TEST  &  EVALUATION 

KD 

PROGRAM  A 

PROGRAM  B 

PROGRAM  C 

TEST  DIVISION 

TEST  DIVISION 

TEST  DIVISION 

Source:  Weapon  System  Acquisition  Guide,  (Air  Command  and  Staff  College,  May  1981) 


Exhibit  C-10.  TYPICAL  TEST  &  EVALUATION  DIRECTORATE  FOR  MULTIPLE  PROGRAM  PO 
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ILS  DIRECTORATE  ORGANIZATION  AND  FUNCTIONS 


O 


•  Provisioning 

•  Planning 

•  Training 

•  Supply  Working  Group 

•  Transportation 

•  Logistics  Program  MGT 


•  Contractor  Support 

•  SATAF 

•  OTfirE  Support 

•  Status  Reporting 

•  Supply  Support 

•  Maintenance  Support 

•  Contractor  Data 

Systems 


•  Fire  Control 

•  Flight  Control 

•  Warranty  Pro¬ 

grams 


•  Procurement 

Data 

•  Engineering 

Data 

•  Support  Equip 

•  ATE 

•  Depot  Support 

•  TCTO’s 


•  Engineering 

•  DR’s 

•  Retrofit 

•  Conf  Mgt 

•  Proprietary 

Rights 

•  Deviations 

Waivers 


•  Engine 

•  Block  A/C 

Changes 

•  A/C  Systems 


•  R&M 

•  LCC/LSA 


Source:  Weapon  System  Acquisition  Guide/  (Air  Command  and  Staff  College,  May  1981) 


Exhibit  C-ll.  TYPICAL  ILS  ORGANIZATION 

f 


* 


APPENDIX  D 


ADDITIONAL  LOGISTIC  SUPPORT  ANALYSIS  INFORMATION 
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APPENDIX  D.  ADDITIONAL  LOGISTIC  SUPPORT  ANALYSIS  INFORMATION 

This  appendix  provides  additional  information  on  Logistic 
Support  Analysis  and  Integrated  Logistic  Support.  It  has  three 
parts.  The  first  is  a  set  of  definitions  of  ILS  elements  (pages 
D-l  to  D-3 ) .  The  second  is  additional  information  on  LSA  task¬ 
ing  (pages  D-4  to  D-ll).  The  third  part  (pages  D-12  to  D-15)  is 
a  list  of  organizations  involved  in  acquisition  logistics  manage¬ 
ment  within  the  Air  Force.  The  sources  for  each  of  the  sets  of 
information  are  noted  on  the  top  of  each  page. 
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1 .  ILS  Elements 


DEFINITIONS 


.  a‘  Maintenance  Planning.  The  process  conducted  to  evolve  *nd  establish 
namtenance  concepts  and  requirements  for  the  lifetime  of  a  materiel  system. 

..  **  Mj».°POwer  and  Personnel.  The  identification  and  acquisition  of 
military  and  civilian  personnel  with  the  skills  and  grades  required  to  operate 
and  support  a  materiel  system  over  its  lifetime  at  peacetime  and  wartime  rates. 

c;  ggpply  Support.  All  management  actions,  procedures,  and  techniques 
used  to  determine  requirements  to  acquire,  catalog,  receive,  store,  transfer 
issue,  and  dispose  of  secondary  items.  This  includes  provisioning  for  initial 
support  as  well  as  replenishment  supply  support. 

.  d‘  g,HPPort  Equipment.  All  equipment  (mobile  or  fixed)  required  to  support 
the  operation  and  maintenance  of  a  materiel  system.  This  includes  associated 
multiuse  end  items,  ground-handling  and  maintenance  equipment,  tools,  metrology 
and  calibration  equipment,  test  equipment,  and  automatic  test  equipment  It 

itself?S  bC  acqU1Sltl0n  of  lo*istics  support  for  the  support  aSd  test  equipment 


e.  Technical  Data.  Recorded  information  regardless  of  form  or  character 
(such  as  manuals  and  drawings)  of  a  scientific  or  technical  nature.  Computer 
programs  and  related  software  are  not  technical  data;  documentation  of  computer 
programs  and  plated  software  are.  Also  excluded  are  financial  data  or  other 
information  related  to  contract  administration. 

tr  **  and  Training  Support.  The  processes,  procedures,  techniques, 

training  devices,  and  equipment  used  to  train  civilian  and  active  duty  and 

inrlul*  ^  t0  op#rat*  and  suPPort  a  -»t«riel  system.  This 

includes  individual  and  crew  training;  new  equipment  training;  initial,  formal 

trIiSe"J°5  training;.and  logistic  support  planning  for  training  equipment  ’ 
and  training  device  acquisitions  and  installations.  4  P 

.  g-  Computer  Resources  Support.  The  facilities,  hardware,  software 

coT£a:;£.r,ou'r-  *nd  p'rso""ei  “eded  to  °p'r“'  -pp»« 

h.  Facilities.  The  permanent  or  semipermanent  real  property  assets 

tvnes^f  farSvf°rt  thC  system,  including  conducting  studies  to  define 

types  of  facilities  or  facility  improvements,  locations,  space  needs,  environ- 
mental  requirements,  and  equipment. 

l*  Handling,  Storage,  and  Transportation.  The  resources 

processes,  procedures,  design  considerations,  and  methods  to  ensure  thatall 
sys  em,  equipment,  and  support  items  are  preserved,  packaged,  handled,  and 
transported  properly,  including  environmental  considerations,  equipment  pre¬ 
servation  requirements  for  short-  and  long-term  storage,  and  transportability. 
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paramete7i7^Ich“I7^&ir'toTreadines10nSd1P  °f  lo8istics"rel«ted  design 
logistics-related  design  parameters  arenexoSP°r5  resoUrce  re9“ir«»ents.  These 
than  as  inherent  values  and  specifically  *“  operational  terms  rather 

objectives  and  support  costs  of  the  Ltenel  system^  "  readiness 

2’  l-nteRrated  Logistics  Snpp^,..  i  di,.,nl .  , 

.nd  to: 
Integrate  support  considerations  into  ays tee  and  eq„lp„ent  deaign. 

readiness  objectives, Pfo  de"g“"”5”“  V«b  “beV'lated  consist'ntly  to 

c.  Acquire  the  required  support. 

d.  Provide  the  retired  support  during  the  operation.!  phase  „  „iniMn 

*"d  ^VetVr“»fV??o"tr^dVVi{,kefd'u"n8eCt“eVe,V,’PliCati0n  °f  ■'*•«*«« 

the  systems  engineering  process,  to  assift  in:  qU1Sltlon  procesa.  as  part  of 
a.  Causing  support  considerations  to  influence  design, 
and  to  each  othej”8  SUPP°rt  re(Juirements  that  are  related  optimally  to  design 


cost. 


c.  Acquiring  the  required  support. 


minimum  cost.  8  ^  required  suPP°rt  during  the  operational  phase  at 

schedule ,  performance^o^systen^readiness^re  CritCrion  established  as  a  cost, 

Decision  Coordinating  Paper  (DCP)  thresholds*** lre“*' nt .  The  term  includes 

“  th'  «itiri.) ““  re<)uiretDents 

nccc^ary.VV^ViVV  CMUnVed  "'nt-  *nd  «»PP»rt  activities 

economical  logistic  support  after  cessaUof  of  /eadiness  objectives  with 
(weapon  system  or  equipment).  f  Production  of  the  end  item 

6*  fieliabilitv-Cent»r»H  Maintenance.  A  c,,^ _ 

preventive  maintenance  tasks  for  an "end  item  v!  appr0ach  for  identifying 

of  “r's/nd  « 

plannedPlogisti«YresIircfsfrincludiIlgCLn^oie0  deSign  characteristics  and 
and  wartime  utilization  requirements.  P°wer»  ®eet  system  peacetime  readiness 

as sorted  with  a  system^during  th^acquisition^h"1  8nd  procurement  c°sts 
ensure  that  planned  support  of  the  weapon  system^s^chieved^  rCqUired  to 
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9‘  gygAeP  Readiness  Objective.  A  criterion  for  assessing  the  abilitv  of  . 
system  to  undertake  and  sustain  a  specified  set  of  missions 

ti«e  mission  capable  rate,  operational  availability,  and  asset  ready  ratT  * 
t*rv^rFiwld‘  i  qua?titative  requirement,  documented  in  the  DCP  and  Secre- 

SsS£25s2^*xisra~- 
,'rSSSs  isss  ks/S 2 
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TABLE  I.  Index  of  Logistic  Support  Analysis  Tasks 


D 

I 

♦c* 


TASK 

SECTION 


PURPOSE  OF 
TASK  SECTION 


TASK/SUBTASK 


INFLUENCE 


SYS/EQUIP 

DESIGN 


SUPPT  SYS 
DESIGN 


LOG  REGMTS 
DETER¬ 
MINATION 


100  -  PROGRAM 
PLANNING  A  CONTROL 


200  -  MISSION 
A  SUPPORT  SYSTEMS 
DEFINITION 


TO  PROVIDE  FOR  FORMAL 
PROGRAM  PLANNING  AND 
REVIEW  ACTIONS 


TO  ESTABLISH 
SUPPORTABILITY 
OBJECTIVES  AND 
SUPPORTABILITY 
RELATED  DESIGN 
GOALS,  THRESHOLDS, 
AND  CONSTRAINTS 
THROUGH  COMPARISON 
WITH  EXISTING  SYSTEMS 
AND  ANALYSES  OF 
SUPPORTABILITY.  COST, 
AND  READINESS  DRIVERS 


101  -  DEVELOPMENT  OF  AN  EARLY  LOGISTIC  SUPPORT 
ANALYSIS  STRATEGY 

101.2.1  -  LSA  STRATEGY 

101.2.2  -  UPDATES 


102  -  LOGISTIC  SUPPORT  ANALYSIS  PLAN 

102.2  1  -  LSA  PLAN 

102.2.2  -  UPDATES 


103  -  PROGRAM  AND  DESIGN  REVIEWS 

103  2  1  -  ESTABLISH  REVIEW  PROCEDURES 
103.2.2  -  DESIGN  REVIEWS 
103  2  3  -  PROGRAM  REVIEWS 
103.2.4  -  LSA  REVIEW 


201  -  USE  STUDY 

201.2.1  -  SUPPORTABILITY  FACTORS 

201.2.2  -  QUANTITATIVE  FACTORS 

201.2  3  -  FIELD  VISITS 

201.2.4  -  USE  STUOY  REPORT  AND  UPDATES 


202  -  MISSION  HARDWARE,  SOFTWARE,  AND 
SUPPORT  SYSTEM  STANDARDIZATION 
202.2.1  -  SUPPORTABILITY  CONSTRAINTS 
202  2.2  -  SUPPORTABILITY  CHARACTERISTICS 

202.2.3  -  RECOMMENDED  APPROACHES 

202.2.4  -  RISKS 


PRIMARY  PURPOSE  OF 
100  SERIES  TASKS 
IS  THE  MANAGEMENT 
AND  CONTROL  OF 
THE  LSA  PROGRAM 


X 

X 

X 

X 


X 

X 

X 

X 


X 

X 

X 

X 


>3C 
*o  •-» 
-s 1— 

— »♦  i 

“I 

#-*o 

»-» I 

w  H- » 
CO 

*QO 

vo  55 

oo  i 

CO  }—* 


X  INDICATES  THAT  THE  SUBTASK  IS  ORIENTED  TOWARD 
INFLUENCING  THE  INDICATED  FACTOR(S). 


TABLE  I.  Index  of  Logistic  Support  Analysis  Tasks.  -  Continued 


TASK 

SECTION 

PURPOSE  OF 
TASK  SECTION 

TASK/SUBTASK 

INFLUENCE  * 

SYS/EQUIP 

OESIGN 

SUPPT  SYS 
OESIGN 

log  regmts 

DETER¬ 

MINATION 

203  -  COMPARATIVE  ANALYSIS 

203.2.1  -  IDENTIFY  COMPARATIVE  SYSTEMS 

X 

X 

203  2.2  -  BASELINE  COMPARISON  SYSTEM 

X 

X 

203.2.3  -  COMPARATIVE  SYSTEM  CHARACTERISTICS 

X 

X 

203.2.4  -  QUALITATIVE  SUPPORTABILITY  PROBLEMS 

X 

X 

203.2.5  -  SUPPORTABILITY,  COST,  AND  READINESS 

DRIVERS 

X 

X 

203.2.6  -  UNIQUE  SYSTEM  DRIVERS 

X 

X 

203  2  7  -  UPDATES 

X 

X 

203.2.8  -  RISKS  AND  ASSUMPTIONS 

X 

X 

204  -  TECHNOLOGICAL  OPPORTUNITIES 

204.2.1  -  RECOMMENDED  DESIGN  OBJECTIVES 

X 

X 

204.2.2  *  UPDATES 

X 

X 

204.2.3  -  RISKS 

X 

X 

205  -  SUPPC  RTABILITY  AND  SUPPORTABILITY  RELATED 

DESIGN  FACTORS 

205  2.1  -  SUPPORTABILITY  CHARACTERISTICS 

X 

205  2  2  -  SUPPORTABILITY  OBJECTIVES  & 

ASSOCIATED  RISKS 

X 

X 

205.2.3  *  SPECIFICATION  REQUIREMENTS 

X 

X 

205.2.4  r  NATO  CONSTRAINTS 

X 

X 

205.2.5  -  SUPPORTABILITY  GOALS  AND 

THRESHOLDS 

X 

X 

MIL-STD-1388-1A 
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TABLE  L  lndex  of  Logistic  Support  Analysis  Tasks.  -  Continued 


TASK 

SECTION 


PURPOSE  OF 
TASK  SECTION 


TASK/SUBTASK 


influence  * 


I  SYS  /EQUIP 
lOESIGN 


300  -  PREPARATION 
AND  EVALUATION 
OF  ALTERNATIVES 


TO  OPTIMIZE  THE  SUPPORT 
[SYSTEM  FOR  THE  NEW  ITEM  , 
AND  TO  DEVELOP  A  SYSTEM 
WHICH  ACHIEVES  THE  BEST 
BALANCE  BETWEEN  COST 
SCHEDULE.  PERFORMANCE. 
AND  SUPPORTABILITY 


301 


FU™?IIONAL  requirements  identification 

301.2.1  FUNCTIONAL  REQUIREMENTS 

UNIQUE  FUNCTIONAL  REQUIREMENTS 
RISKS 

OPERATIONS  AND  MAINTENANCE  TASKS 
DESIGN  ALTERNATIVES 
UPDATES 


301.2  2 

301.2.3 

301.2.4 

301.2.5 

301.2.6 


302  -  SUPPORT  SYSTEM  ALTERNATIVES 

302.2.1  ALTERNATIVE  SUPPORT  CONCEPTS 
SUPPORT  CONCEPT  UPDATES 
ALTERNATIVE  SUPPORT  PLANS 
SUPPORT  PLAN  UPDATES 
RISKS 


302.2.2 

302.2.3 

302.2.4 

302.2.5 


303  -  EVALUATION  OF  ALTERNATIVES  AND 
TRADEOFF  ANALYSIS 
303.2  1  TRADEOFF  CRITERIA 

SUPPORT  SYSTEM  TRADEOFFS 
SYSTEM  TRADEOFFS 
READINESS  SENSITIVITIES 
MANPOWER  AND  PERSONNEL  TRADEOFFS 
TRAINING  TRADEOFFS 
REPAIR  LEVEL  ANALYSES 
DIAGNOSTIC  TRADEOFFS 
COMPARATIVE  EVALUATIONS 
ENERGY  TRADEOFFS 
SURVIVABILITY  TRADEOFFS 
TRANSPORTABILITY  TRADEOFFS 


303.2.2 

303.2.3 

303.2.4 

303.2.5 

303.2.6 

303.2.7 

303.2.8 

303.2.9 

303.2.10 

303.2.11 
303  2.12 


SVS  ^ 

OESIGN  Iminatiom 


( 
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TABLE  I.  Index  of  Logistic  Support  Analysis  Tasks.  -  Continued 


TASK 

SECTION 

PURPOSE  OF 
TASK  SECTION 

TASK/SUBTASK 

INFLUENCE  * 

SYS/EQUIP 

DESIGN 

SUPPT  SYS 
DESIGN 

LOG  REGMTS 
DETER. 

400  •  DETERMINATION  OF 

TO  IDENTIFY  THE  LOGISTIC 

401  •  TASK  ANALYSIS 

LOGISTIC  SUPPORT 

SUPPORT  RESOURCE 

401.2.1  TASK  ANALYSIS 

V 

RESOURCE  REQUIREMENTS  / 

REQUIREMENTS  OF  THE 

401.2.2  ANALYSIS  DOCUMENTATION 

x 

NEW  SYSTEM  IN  ITS 

401.2.3  NEW/CRITICAL  SUPPORT  RESOURCES 

x 

OPERATIONAL 

401.2.4  TRAINING  REQUIREMENTS  AND 

ENVIRONMENT(S) 

RECOMMENDATIONS 

X 

AND  TO  DEVELOP  PLANS 

401.2.5  DESIGN  IMPROVEMENTS 

x 

x 

FOR  POST  PRODUCTION 

401.2  6  MANAGEMENT  PLANS 

x 

SUPPORT 

401.2.7  TRANSPORTABILITY  ANALYSIS 

x 

x 

401.2  8  PROVISIONING  REQUIREMENTS 

x 

401.2.9  VALIDATION 

x 

x 

x 

401.2.10  ILS  OUTPUT  PRODUCTS 

x 

401.2.11  LSAR  UPDATES 

X 

X 

X 

402  -  EARLY  FIELDING  ANALYSIS 

402  2.1  NEW  SYSTEM  IMPACT 

X 

402.2.2  SOURCES  OF  MANPOWER  AND 

PERSONNEL  SKILLS 

X 

402.2.3  IMPACT  OF  RESOURCE  SHORTFALLS 

X 

402  2.4  COMBAT  RESOURCE  REQUIREMENTS 

X 

402  2  5  PLANS  FOR  PROBLEM  RESOLUTION 

X 

403  -  POST  PRODUCTION  SUPPORT  ANALYSIS 

403.2  POST  PRODUCTION  SUPPORT  PLAN 

X 

X 

500  -  SUPPORT  ABILITY 

TO  ASSURE  THAT  SPECIFIED 

501  -  SUPPORT  ABILITY  TEST.  EVALUATION. 

ASSESSMENT 

REQUIREMENTS  ARE 

AND  VERIFICATION 

ACHIEVED  AND  DEFICIENCIES 

501.2.1  TEST  AND  EVALUATION  STRATEGY 

x 

x 

x 

CORRECTED 

501.2.2  OBJECTIVES  AND  CRITERIA 

X 

X 

x 

501.2.3  UPDATES  AND  CORRECTIVE  ACTIONS 

x 

x 

x 

501.2.4  SUPPORTABILITY  ASSESSMENT  PLAN 

(POST  DEPLOYMENT) 

X 

X 

X 

501.2.5  SUPPORTABILITY  ASSESSMENT 

(POST  DEPLOYMENT) 

X 

X 

X 
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MMMMAM 

DOCUMENTATION 

BASELINE 


SPECIFICATIONS 


REVIEWS  S  AUDITS 


ILS  PLANNING 


LOGISTIC 

SUPPORT 

ANALYSIS 

EFFORT 


PRE-CONCEPT 

CONCEPT  EXPLORATION 

DEMONSTRATION  1  FULL  SCALE 

ANO  VAU0ATI0N  |  DEVELOPMENT 

PRODUCTION  /DEPE0TMENT/ 
POST  PRODUCTION 

JUSTIFICATK 

SYSTEM  l 

3N  FOR  MAJOR  SYSTEM  COR 

NEW  START  SYSTEM  COR 

FUNCTI 

ICEPT  PAPER  DECISION  COOK 

INTEGRATED  Pf 

ONAL  ALLO< 

SYSTEM  SPECIFICATION 

lOINATION  PAPER  t 

IOGRAM  SUMMARY 

LtED  PRO 

DUCT 

SYSTEM 

REQUIREMENT 

DEVELOPMENT  SPECIFICATION 

- - -  ► 

SYSTEM  DESIOI 
REVIEW 

PROOUCT/PROCESS/ ^ 

MATERIAL  SPECIFICATION 

4  PRELIMINARY  CRITICAL  I 

DESIGN  REVIEW  "ev**w  , 

DC  MON  FORMAL  QUALIFICATIONS 

RCV1CW 

1 

TEST  A 
EVALUATION 


LOGISTICS  OUTPUTS 
FOR  SUBSEQUENT 
PHASE 


DEVELOPMENT  OF  AN  EARLY 
LOGISTIC  SUPPORT 
ANALYSIS  STRATEGY  (101) 

USE  STUDY  (201) 

COMPARATIVE  ANALYSIS  (203) 


ILS  PLAN/INTEGRATED  SUPPORT  PLAN 


DEVELOPMENT  OF  AN  EARLY 
LOGISTIC  SUPPORT 
ANALYSIS  STRATEGY  (101) 

LOOISTIC  SUPPORT 
ANALYSIS  PLAN  (102) 

PROGRAM  AND  DESIGN 
REVIEWS  (103) 

USE  STUDY  (201) 

MISSION  HAR0WARE. 
SOFTWARE,  AND 
SUPPORT  SYSTEM 
STANDARDIZATION  (202) 

COMPARATIVE  ANALYSIS  (203) 

TECHNOLOGICAL 
OPPORTUNITIES  (204) 

SUPPORT  AB  H_  I TY  AND 
SUPPORT  ABILITY  RELATED 
DESIGN  FACTORS  (205) 

FUNCTIONAL  REQUIREMENTS 
IDENTIFICATION  (301) 

SUPPORT  SYSTEM 
ALTERNATIVES  (302) 

EVALUATION  OF  ALTERNATIVES 
ANO  TRADEOFF  ANALYSIS  (303) 

SUPPORTABILfTY  TEST, 
EVALUATION  AND 
VERIFICATION  (SOI) 


t 


SUPPORT  ABILITY  CONSTRAINTS 
LSA  STRATEGY 


DEVELOPMENT  OF  AN  EARLY 
LOGISTIC  SUPPORT 
ANALYSIS  STRATEGY  (101) 

LOGISTIC  SUPPORT 
ANALYSIS  PLAN  (102) 

PROORAM  ANO  DESIGN 
REVIEWS  (103) 

USE  STUOY  (201) 

MISSION  HARDWARE. 
SOFTWARE  AND 
SUPPORT  SYSTEM 
STANDARDIZATION  (202) 

COMPARATIVE  ANALYSIS  (203) 
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FIGURE  2.  Logistic  Support  Analysis  Process  Overview. 
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APPENDIX  E 

GUIDANCE  ON  DEVELOPING  A  SCHEDULING  NETWORK* 


*The  information  contained  in  this  appendix  has  been  extracted 
verbatim  from  the  on-line  Users  Guide  for  the  Computer  Supported 
Network  Analysis  System  (CSNAS),  owned  and  operated  by  AFALC/XRI . 
For  more  information  on  this  system,  contact  Mr.  Albert  L. 

Clark,  AFALC/XRI,  Wright-Patterson  AFB ,  OH.  Autovon  785-6725 
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APPENDIX  E.  GUIDANCE  ON  DEVELOPING  A  SCHEDULING  NETWORK 

This  appendix  provides  a  brief  plan  for  developing  a  program 
schedule  network.  It  has  been  extracted  intact  from  the  on-line 
Users  Guide  for  the  Computer  Supported  Network  Analysis  System 
( CSNAS ) ,  owned  and  operated  by  AFALC/XRI .  While  largely  intended 
to  assist  Deputy  Program  Managers  in  Logistics  in  planning  their 
logistics-related  activities,  the  model  and  the  associated  gui¬ 
dance  quoted  in  this  appendix,  may  be  of  larger  use  to  the  PM  and 
his  staff  in  balancing  materiel  readiness  and  concurrency. 
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NETWORK  ANALYSIS 


WHY 


1 .  PURPOSE 

The  purpose  of  network  analysis  is  to  develop  a  logical 

1 

schedule,  determine  the  critical  path,  identify  which  jobs  have 
how  much  slack  time,  and  to  integrate  the  separate  schedules  of 
all  portions  of  a  project.  Without  network  analysis,  the  manager 
is  not  integrating  the  portions  of  a  project,  he  has  a  tendency 
to  think  the  wrong  jobs  are  critical  to  the  schedule  and  has 
everyone  working  in  the  wrong  direction  while  time  slips  away. 

a.  The  program/project  manager  does  not  understand  how 
everything  interrelates  and  how  a  slip  in  one  area  may 
or  may  not  effect  the  overall  schedule. 

b.  The  submanager  (SE  manager,  Engir.eer,  Contractor,  etc.) 
may  think  that  his  job  should  have  all  the  attention  at 
a  point  in  time  when  his  area  has  time  to  burn.  When 
all  the  submanagers  think  their  problem  is  critical  and 
when  there  is  no  network  analysis  the  project  manager 
may  be  swayed  by  personality  versus  analysis  of  the 
problem. 

c.  The  submanager  has  a  tendency  for  tunnel  vision  and 
think  he  has  plenty  of  time  to  do  a  portion  of  the  job 
and  not  realize  that  someone  else  cannot  start  until  he 
has  finished.  The  project  manager  may  be  lashing  the 
wrong  person  for  "why  the  hold  up". 


2.  DESIRED  RESULTS 

A  network  analysis  should  graphically  show  all  concerned 
with  a  project  just  where  each  individual  job  fits  into  the  whole 
project.  Just  the  diagram  with  no  dates  or  times  can  be  invalu¬ 
able  to  management,  supervision,  and  worker.  The  logic  diagram 
is  the  hardest  part  and  that  is  why  model  networks  have  been 
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developed.  If  the  program  manager/DPML  cannot  develop  the  logic 
diagram  of  what  jobs  are  done  in  what  sequence,  how  can  anyone 
possibly  expect  to  have  a  good  schedule. 

a.  The  network  analysis  logic  diagram  is  worth  a  thousand 
words.  It  is  this  diagram  that  is  a  main  integrator 
between  ILS  and  the  program,  between  ILS  elements, 
between  the  contractor  and  between  all  of  the  USAF 
commands  associated  with  a  project.  An  ILSP  without  a 
network  analysis  is  only  the  binding  together  of  a 
series  of  separate  sub  plans.  the  "I"  in  Integrated 
Logistics  Support  (ILS)  is  more  than  a  compilation  of 
plans.  The  network  analysis  shows  how  all  of  these 
separate  entities  interrelate  in  an  integrated  schedule 

b.  The  network  analysis  provides  more  than  the  typical 
milestone  chart  that  is  a  series  of  guesses  as  to  when 
jobs  should  be  done.  At  worst  a  milestone  chart  is  the 
product  of  when  the  contractor  is  going  to  do  things 
versus  our  schedule  of  when  we  want  things  done  by  not 
only  the  contractor  but  everyone  with  a  major  role. 
Everyone  should  be  marching  to  our  plan  versus  doing 
their  own  thing  and  getting  approval  to  vary  from  our 
schedule.  The  contractor  and  everyone  else  on  the 
project  should,  of  course,  be  involved  in  the  develop¬ 
ment  of  the  schedule.  Without  program  guidance,  the 
contractor  may  be  delaying  an  item  critical  to  the 
program  without  realizing  what  he  is  doing.  The 
manager  of  a  project  must  also  be  the  planner  and  the 
leader  to  make  things  happen. . .versus  just  reactina  to 
disasters. 

c.  A  network  analysis  tells  us  what  jobs  are  on  the  criti¬ 
cal  time  path.  This  critical  path  tells  us  what  jobs 
must  stay  on  schedule  and  which  jobs  can  slip,  how 
much.  A  network  analysis  enables  us  to  develop  a  job 
logic  and  an  estimated  time  to  do  each  job  and  provides 
us  with  an  earliest  start  and  finish  time,  a  latest 
possible  start  and  finish  time,  the  amount  of  slack 
time  on  any  job,  and  the  critical  time  path. 

d.  With  network  analysis  the  manager  has  a  logical  plan, 
can  control  time  better,  and  hopefully,  the  manager  can 
deliver  a  supportable  system  in  less  time  with  equal  or 
lower  cost. 
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NETWORK  ANALYSIS 


HOW 

1.  STEP  ONE:  ESTABLISH  PURPOSE  OF  THE  PLAN: 

You  must  decide  what  is  this  network  for,  who  is  it  for,  how 
will  it  be  used.  If  it  is  for  the  program  manager,  only  go  to 
the  level  of  detail  that  he  needs  to  manager.  If  the  network  is 
intended  to  be  the  very  top  level  schedule  of  key  jobs ...  flight 
number  6  to  deliver  5%  of  spares  does  not  belong  on  the  top  level 
production  and  deployment  network.  It  might  belong  on  a  sub¬ 
network  to  show  the  details  of  site  activation  number  7.  Site 
activation  number  7  may  be  a  task  on  a  sub-network  to  show  the 
details  of  a  top  level  task  on  the  top  levele  network  that  is 
called  "deploy  system" .  When  the  program  manager  is  developing  a 
schedule  to  show  how  he  will  get  a  system  from  FSED  to  IOC  he  may 
want  to  know  the  time  frame  of  when  deployments  will  occur  but  he 
is  not  ready  to  look  at  the  details  of  when  each  operational  site 
will  be  activated  let  alone  when  flight  number  6  is  arriving.  In 
fact  the  only  one  to  worry  these  details  will  probably  be  the 
deployment  manager ...  only  if  he  has  a  problem  will  he  tell  the 

dpml/pm. 

EXAMPLE:  Sub-network  for  installation  of  ATE  test  set. 

2.  STEP  TWO:  TIME  FRAME  OF  THE  PLAN: 

You  must  decide  how  much  you  are  ready  to  plan  for.  You 
should  have  a  network  or  a  series  of  networks  to  cover  all  of 
FSED  if  you  are  going  to  send  out  an  RFP  for  all  of  FSED.  If  by 


chance  you  hve  already  gone  on  contract  or  already  sent  out  that 
RFP  it  is  essential  that  your  networks  cover  this  period. 

You  many  want  to  consider  including  the  network  analysis  as 
part  of  the  FMP ,  ILSP,  and  RFP .  The  RFP  should  then  call  for  the 
contractor  to  respond  to  this  schedule  with  what  is  wrong  with 
it,  how  he  will  integrate  with  it,  and/or  what  his  proposed  net- 
analysis  indicates  instead  of  your  schedule.  A  network 
attached  to  his  ISP  would  show  how  he  proposes  to  integrate 
logistic  support.  This  network  must  match  our  plans  before  we 
actually  sign  the  contract.  Schedule  deviations  should  be  justi¬ 
fied  by  the  responsible  party,  approved  by  the  manager,  changed 
in  the  contracts  and  the  networks.  Results  of  the  network 
changes  should  be  provided  to  all  concerned  to  adjust  their  own 
networks/plans  accordingly. 

EXAMPLE:  This  sub-network  will  begin  at  SERD  submittal  and  end 

when  the  ATE  is  installed  at  location  one. 

3.  STEP  THREE:  LIST  THE  JOBS  (TASKS)  TO  BE  DONE. 

You  now  get  all  of  the  key  managers  involved  in  making  a 
list  of  the  tasks  that  must  be  done  to  meet  step  one  and  step  two 
above.  Don't  over  work  this  area  as  later  steps  will  help  you 
find  more. 

1.  IDENTIFY  SUPPORT  EQUIPMENT 

2.  ORDER  SUPPORT  EQUIPMENT 

3.  LEADTIME  FOR  SUPPORT  EQUIPMENT 

4 .  INSTALL  SUPPORT  EQUIPMENT 


EXAMPLE : 


4.  STEP  FOUR:  DEVELOP  LOGIC/SEQUENCING. 

You  must  now  copy  all  of  these  jobs  listed  in  step  three 
onto  slips  of  paper.  You  can  start  at  either  the  front  or  back 
of  a  problem.  We  recommend  that  you  start  with  finish  and  work 
backwards.  Use  a  blank  wall,  a  large  table  top,  a  sheet  of  but¬ 
cher  paper,  etc.  Put  up  a  task  slip  on  the  far  right  hand  side 
that  says  "finish".  Put  up  the  task  slip  that  appears  to  be  the 
last  job  required  before  the  finish  of  the  network.  Now  ask  the 
group,  "is  there  any  other  job  that  must  be  done  after  this  job 
and  before  we  have  finished?".  When  this  has  been  done  put  up 
the  next  task  slip  representing  some  job  that  is  done  prior  to 
the  last  job  you  put  up  and  ask  the  questions ...  is  the  last  job 
dependent  upon  this  job  in  any  way,  is  there  any  job  that  must  be 
done  after  this  one  but  before  the  last  job????.  Repeat  this 
process  until  you  have  worked  your  way  back  to  today.  If  your 
plan  starts  growing  in  size  and  becoming  overly  complex,  take  a 
look  at  the  job  descriptions  again  to  make  sure  that  you  are 
sticking  to  your  decisions  in  step  one  and  two.  Are  you  getting 
into  too  much  detail?  Is  each  task  of  sufficient  importance  to 
be  included  on  this  plan  or  does  it  apply  more  to  lower  level 
management?  Remember  that  the  ultimate  limit  is  400  tasks  and 
anything  over  200  is  very  difficult  to  understand.  Draw  lines 
between  dependent  tasks  from  finish  to  start.  If  one  job  can 
start  when  another  is  half  complete  break  the  predecessor  job 
into  two,  half  complete  and  all  complete.  Add  an  OPR  for  each 
job  to  show  who's  responsible.  If  your  network  is  too  big  you 
may  have  tried  to  cover  too  long  a  period  of  time  or  gotten  into 
too  much  detail. 
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5.  STEP  FIVE:  TIME  THE  JOBS/TASKS 

You  can  load  the  network  into  the  computer  with  zero  time 
tasks.  If  you  are  just  working  out  the  logic  and  are  not  ready 
to  consider  work  times  we  suggest  that  you  use  a  time  of  one  time 
unit  for  each.  That  way  the  task  duration  time  becomes  at  least 
a  counter  and  will  show  the  most  numbe  of  jobs  on  a  path  as  the 
critical  path.  If  you  use  zero  time  on  all  the  jobs,  the  com¬ 
puter  will  say  everything  must  happen  at  once  and  everything  is 
critical . 

The  time  we  are  referring  to  is  a  job  duration  time  for  each 
job  on  the  network.  The  software  will  accept  either  work  days 
based  on  a  5  day  workweek  and  computing  in  weekends  or  work 
weeks.  If  all  of  your  networks  are  somehow  related  or  will  be 
they  should  all  be  the  same,  days  or  weeks.  All  model  networks 
are  in  weeks,  because  over  a  planning  period  of  years  you  will  be 
lucky  to  come  within  weeks  of  original  planning  let  alone  hit  an 
exact  day. 

There  are  several  ways  to  arrive  at  a  task  duration  time: 


a . 

Contractual  time 

b. 

Task  time  =  optimistic 

+  (4  x  most 

likely)  + 

pessimistic 

c . 

Experience 

6 

d. 

We  recommend  your  best 
changing  task  times  as 
wrong . 

educated  guess  and  plan  on 
soon  as  you  learn  you  guessed 
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e.  If  you  want  a  planned  schedule  and  a  pessimistic  sched¬ 
ule,  simply  do  one  schedule  and  then  copy  that  network 
onto  another  computer  file  and  change  the  task  times  on 
the  new  file. 

YOU  ARE  NOW  READY  TO  LOAD  YOUR  NETWORK  INTO  THE  COMPUTER  TO 
COMPUTE  START  AND  FINISH  DATES,  CRITICAL  PATH,  AND  SLACK 
TIME. 

The  software  has  an  AFALC/XR  imposed  limit  of  400  tasks  per  net¬ 
work.  However,  the  user  should  limit  the  number  of  tasks  to 
100-150.  This  limit  is  imposed  because  networks  should  be  small 
enough  for  personnel  to  comprehend.  A  second  reason  is  because 
networks  on  a  project  ae  meant  to  be  update  regularly  to  reflect 
the  varius  changes  that  always  occur  during  a  project.  If  a  net¬ 
work  is  very  large  it  takes  a  long  time  to  print;  it  won't  hang 
easily  on  a  wall  where  it  can  be  referred  to  regularly;  and  con¬ 
sequently  it  falls  into  disuse  where  it  does  not  do  anything  or 
do  any  good  for  program  management.  Good  networking  dictates 
that  you  should  start  with  a  top  level  program  network  of  a 
limited  number  of  high  level  tasks  that  top  level  management  is 
actually  interested  in  monitoring.  A  top  level  task  may  than 
become  the  summation  of  a  lower  level  network. 

EXAMPLE:  Task  number  3100  on  the  top  level  network  "ILST&E" 

becomes  network  file  number  T3100N  when  file  T3100N  is 
run  through  the  software  it  prints  the  detailed  tasks 
required  to  do  "ILST&E. 

There  have  been  many  programs  that  had  an  automated  network 
of  7,000-10,000  Tasks  that  got  lost  in  detail.  A  low  level  task 
or  series  of  low  level  tasks  were  changed  causing  changes  to  the 
top  level  network  tasks  and  the  reasons  got  lost... The  program 
got  off  schedule ...  and  control  of  time  was  lost... The  most  impor¬ 
tant  aspect  of  network  analysis.  Hence  the  user  should  have 
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sub-networks  of  400  or  fewer  tasks  that  require  each  network  to 


be  updated  independently  of  other  networks  until  management  has 
approved  the  changes  and  caused  the  input  to  each  level  of  network. 
Rather  than  computing  sub-networks  together  as  a  single  net¬ 


work  for  automatic  update  of  all  levels,  this  network  technique 


calls  for  having  each  network  be  independent  but  related.  If  the 
level  one  network  has  a  task  number  5000  "DELIVER  FIRST 


AIRCRAFT 


AIRCRAFT",  then  the  dates  for  this  task  become  mandatory  on  lower 
level  networks - CONTROL...  If  the  manager  of  level  two  network 


cannot  meet  this  given  date  for  task  number  5000  then  he  should 
brief  the  level  one  network  manger  as  to  why  not  and  how  far  he 
will  miss  this  date.  The  level  one  manager  must  then  either 
agree  to  input  a  change  to  task  number  5000  on  the  level  one  net 
work  or  force  the  manager  of  level  two  network  to  go  back  and 
work  on  his  plans  to  try  to  meet  the  mandatory  dates  on  task 
number  5000 . 

Network  analysis/program  control  takes  work. .. Automation 
helps  only  to  a  point .. .beyond  which ...  control  is  lost. 
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